| 3 | | The background issue should now be solved for good. There were two issues the contributed to this problem. The first one was the value being written to disk was being trucated via an effective int() call, which isn't appropriate around zero. To make this fully consistent, psLib now takes the floor() of the value. The second issue related to the fuzz value, and I was confused about the math in the 2014-05-16 plot that implied that a fuzz value of 1.0 was needed. Gene pointed out on Monday that the floor function effectively removes a random value between 0 and 1 for all numbers fed into it. To correct for this, the fuzz value needs to add back this random value, corresponding to a fuzz value of zero. Due to the int vs floor vs ceil, this is mathematically equivalent to what I was finding (int for negative numbers is ceil, floor and ceil are offset by 1 unit). |
| | 3 | The background issue should now be solved for good. There were two issues the contributed to this problem. The first one was the value being written to disk was being trucated via an effective int() call, which isn't appropriate around zero. To make this fully consistent, psLib now takes the floor() of the value. The second issue related to the fuzz value, and I was confused about the math in the 2014-05-16 plot that implied that a fuzz value of 1.0 was needed. Gene pointed out on Monday that the floor function effectively removes a random value between 0 and 1 for all numbers fed into it. To correct for this, the fuzz value needs to add back this random value, corresponding to a fuzz value of zero. Due to the int vs floor vs ceil, this is mathematically equivalent to what I was finding previously (int for negative numbers is ceil, floor and ceil are offset by 1 unit). |