Fix HalfTypeHelper for Infinity and NaN - #613
Conversation
|
The removal of uif makes sense, but it'd be good to have an explanation of what range of values changed - the test app just needs more samples, that's all! |
|
For float s e m. My e range is [-127, 128] The change range is:
|
|
My change mean: Note: sign should be 0 or 1. previous code is not correct. |
|
Previous code with if branch float.Infinity. e=128 when Infinity. In fact, when e>16, all is the same regardless float is Infinity. |
|
I know what happen now. 32bit float to 16bit float: float f = 5f;
// rounding 23bit mantissa to 10bit mantissa
if (f > HalfSingle.MaxValue)
{
return HalfSingle.MaxValue;
}
else if (f < HalfSingle.MinValue)
{
return HalfSingle.MinValue;
}HalfSingle.MaxValue = 0x7FFF; // s=0, e=16, m=0x3FFF |
|
In the transform, 32bit float should be regarded as no NaN and Infinity too. |
|
Test case: using Microsoft.Xna.Framework.Graphics.PackedVector;
using System;
namespace ConsoleApp5
{
internal static class Show
{
[STAThread]
private static void Main(string[] args)
{
Console.WriteLine("PositiveInfinity f32->f16: 0x" + new HalfSingle(float.PositiveInfinity).PackedValue.ToString("X"));
Console.WriteLine("PositiveInfinity f16->f32: " + F16(0, 16, 0)/*HalfSingle.PositiveInfinity*/.ToSingle());
Console.WriteLine("NaN f32->f16: 0x" + new HalfSingle(float.NaN).PackedValue.ToString("X"));
Console.WriteLine("NaN f16->f32: " + F16(1, 16, 0x200)/*HalfSingle.NaN*/.ToSingle());
}
// e=[-127, 128] m=[0, 0x7FFFFF]
private static unsafe float F32(int s, int e, int m)
{
int i = s << 31 ^ e + 127 << 23 ^ m;
return *(float*)&i;
}
// e=[-15, 16] m=[0, 0x3FF]
private static HalfSingle F16(int s, int e, int m)
{
HalfSingle hf = default;
hf.PackedValue = (ushort)(s << 15 ^ e + 15 << 10 ^ m);
return hf;
}
}
}XNA output: FNA output: For IEEE, 0x7C00 mean HalfSingle.PositiveInfinity, 0xFE00 mean HalfSingle.NaN. For IEEE, 0x7FFF mean HalfSingle.NaN, 0xFFFF mean HalfSingle.NaN. It mean float.PositiveInfinity not map to HalfSingle.PositiveInfinity. It mean HalfSingle.NaN and HalfSingle.PositiveInfinity transform to a normal float value. I think that for XNA, Infinity and NaN is meanless. So ignore it. |
|
How do you think? @flibitijibibo No need process infinity and NaN. It is the same behavior as XNA. The code for f16 to f32 in FNA not process infinity and NaN. Why process them for f32 to f16 in FNA? |
|
The tests look good to me, so will defer this to someone who's better at math - @thatcosmonaut should be able to check this out when he's back next week. |
The same behavior as XNA.
Test case: