Document select() failure with buffered file

[rongarret]

BPO	1705393
Nosy	@amauryfa, @devdanzin

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2007-04-22.20:38:27.000>
labels = ['easy', 'type-bug', '3.9', '3.10', '3.11', 'docs']
title = 'Document select() failure with buffered file'
updated_at = <Date 2021-12-05.16:59:15.125>
user = 'https://bugs.python.org/rongarret'

bugs.python.org fields:

activity = <Date 2021-12-05.16:59:15.125>
actor = 'iritkatriel'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2007-04-22.20:38:27.000>
creator = 'rongarret'
dependencies = []
files = []
hgrepos = []
issue_num = 1705393
keywords = ['easy']
message_count = 6.0
messages = ['31865', '31866', '31867', '84716', '85311', '116665']
nosy_count = 5.0
nosy_names = ['amaury.forgeotdarc', 'ggenellina', 'rongarret', 'ajaksu2', 'docs@python']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue1705393'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

[rongarret]

select.select fails on file-like objects created by socket.makefile when data exists in the file buffer but not in the underlying socket.

Here is code to reproduce the bug. Run it, then point a web browser at port 8080 and observer that select times out indicating that no input is available even though input is still in fact available.

=======

from SocketServer import *
from socket import *
from select import select

class myHandler(StreamRequestHandler):
  def handle(self):
    while 1:
      sl = select([self.rfile],[],[],1)[0]
      print sl
      l = self.rfile.readline()
      if len(l)<3: break
      print l,
      pass
    print>>self.wfile, 'HTTP/1.0 200 OK'
    print>>self.wfile, 'Content-type: text/plain'
    print>>self.wfile
    print>>self.wfile, 'foo'

server = TCPServer(('',8080), myHandler)
server.serve_forever()

[ggenellina]

The handler should not do its own select() call. (And even then, select should use the underlying socket, not the wrapping pseudo-file object).
Just removing the select() call works fine.

[rongarret]

Just removing the select() call works fine.

For this simple example yes, not for more complicated cases. My particular application is a transparent HTTP proxy which needs to be able to serve multiple connections at once and handle keep-alive connections. (The proxy connects to local server processes through unix sockets, which is why I can't use a regular HTTP proxy. Send me email if you want more details about why I'm doing this.) Without writing a full-blown HTTP-aware proxy, select (or something like it) is necessary.

Upon further study, though, I have come to the conclusion that this is not actually a bug (since select is doing what it is advertised to do), just very counter-intituitive behavior. If select is going to accept file objects as arguments it ought to do the Right Thing with them IMO.

[devdanzin]

Cannot verify for trunk.

[amauryfa]

This older post
http://bytes.com/groups/python/786579-python-2-2-1-select
describes a similar problem where select() is used on a buffered file
object (a pipe to another process)

IMO it should be documented that select() does not work so well for
objects which have a fileno(), but do data buffering: select() only sees
the file descriptor and cannot know whether there is data in the buffer.

[BreamoreBoy]

I've changed things in reply to msg85311, feel free to alter things again if you disagree.