Fix hang on UTF-16 LE BOM file

[chrchr-github]

fixes #637

[firewave]

Yeah - there's several places which lack checks for EOF. I have WIP tests for that and other corner cases/gaps somewhere - something to be done in preparation of #566.

[firewave]

With #647 it should be possible to write a test which uses the input in question.

[chrchr-github]

With #647 it should be possible to write a test which uses the input in question.

Can we just run the tests with all three options?

[firewave]

Can we just run the tests with all three options?

Yes, if somebody adds the functionality. It didn't seem straight forward to me so I haven't yet. Feel free to do so.

[firewave]

I think I took a wrong turn. That should trigger with regular unit test. So please add a test for it.

[firewave]

I think I took a wrong turn. That should trigger with regular unit test. So please add a test for it.

Ah - you mentioned FileStream in the ticket. Still I think we should have a unit test (if missing).

[chrchr-github]

So there is also some kind of platform divergence going on...

[chrchr-github]

These (hanging) runs may be cancelled by the way:
https://github.com/cppcheck-opensource/simplecpp/actions/runs/25805772896
https://github.com/cppcheck-opensource/simplecpp/actions/runs/25805772942/job/75807948771

[chrchr-github]

So there is also some kind of platform divergence going on...

Of course f.write("\xFF\xFE\x3B\x00") does something different on Linux, because why wouldn't it?

~$ hexdump pytest.cpp 
0000000 bfc3 bec3 003b                         
0000006

[danmar]

I guess CI will be fixed by #641

[danmar]

I believe the input to std::isalnum must be 0-0xff

[danmar]

hmm the documentation says:

the behavior of std::isalnum is undefined if the argument's value is neither representable as unsigned char nor equal to EOF.

so we are safe right?

[chrchr-github]

hmm the documentation says:

the behavior of std::isalnum is undefined if the argument's value is neither representable as unsigned char nor equal to EOF.

so we are safe right?

That's how I read it too.