read tcp i/o timeout on Windows 10

[hexadecy]

After some db calls, we have to restart our Go backend:

Message: read tcp 127.0.0.1:50743->127.0.0.1:27017: i/o timeout

MongoDB 3.4.10
Go 1.9.2 windows/amd64
github.com/globalsign/mgo 5be15cc

I think it was ok with:
MongoDB 3.2.16
github.com/globalsign/mgo c4a7121
Go 1.9.1 Alpine 3.6

[domodwyer]

Hi @hexadecy - thanks for the report.

Do you have any more info?

• Do all requests fail after a certain point?
• What timeouts do you have configured?
• Are you using bulk ops? What sort of workload are you running?
• Do you see the issue with 5be15cc and Mongo 3.2 or only on 3.4?

I'll get a new test running in our environment ASAP.

Dom

[hexadecy]

• Yes all requests fail after the timeout, we use mgodb.Session.Clone() for all of our http handlers
• Default timeouts, our project use nginx 1.12.2 as a proxy and labstack/echo v3 as go framework
• No bulk ops, just single user workload on our dev environment
• We only tested 3.4 so far with this commit 5be15cc

[domodwyer]

Thanks @hexadecy - I'll look into it ASAP.

[hexadecy]

That change looks suspicious to me:
5be15cc#diff-a75970bbdfa9237a3958512f8cc2fb09

Why have you reverted it than merged it?
We only have this issue on two Windows pc so far.

[domodwyer]

Hey @hexadecy

I've been running a test against one of our mongo environments (replicated & sharded) over the last 24 hours and I'm unable to reproduce the problem against 3.2.x. I also killed all connections at random throughout the night, and they recovered automatically.

I'll switch to 3.4 and try another run, but I don't think the change between 5be15cc and c4a7121 is likely to behave differently between the two versions - I might be wrong though!

For reference I'm using this load tool with 5be15cc to push mongo - does https://github.com/domodwyer/mpjbt/blob/1d8068910f0d1972947c6523e1093cc03ebb6343/mongo/db_opts.go#L29 look like your session.Clone() usage? Are you clients connecting directly to mongod or through mongos (or any firewalls/routers)?

Dom

[domodwyer]

It was reverted as it incorrectly targeted master (see #54) - we merge to development, test in staging and then merge to master - sorry for the confusion!

[weiishann]

Hi @hexadecy,

How are you using Nginx as a proxy? Could you elaborate please?

Thanks

[hexadecy]

• We use no sharding right now and it's direct localhost mongod
• Except in Qa / Prod we use docker-compose network
• Nginx is used only for the client, to mix Node.js and Go. They both talk to Mongo directly (localhost:27017).

The Go net.Conn implementation for Windows use "wsasend" and "wsarecv"
https://golang.org/src/net/fd_windows.go

But maybe they are not that thread safe:
http://www.serverframework.com/asynchronousevents/2015/01/wsarecv-wsasend-and-thread-safety.html

[domodwyer]

So it looks like it's probably a windows-specific issue - we're running tests against 3.4.x anyway just to eliminate it as a possible cause.

We're going to look into this - in the meantime I suggest vendoring 5be15cc if you're using windows.

Dom

[hexadecy]

Ok, it happens again on the same Win pc.
We will try c4a7121 and mongo 3.4.10

From an old thread:
https://groups.google.com/forum/#!topic/golang-nuts/inm0Bu_FDk4

Socket unlock was always after as far as I know (2010)

_, err = socket.conn.Write(buf)
socket.unlock