Extract file-finding and language-handling code from gettext.find

[jjinux]

BPO	1649329
Nosy	@loewis, @gpshead, @devdanzin, @tarekziade, @chrysn, @merwok
Files	gettext.patch: Patch showing a possible way to refactor the code

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2007-02-01.02:20:07.000>
labels = ['3.7', 'type-feature', 'library']
title = 'Extract file-finding and language-handling code from gettext.find'
updated_at = <Date 2017-09-27.21:43:24.307>
user = 'https://bugs.python.org/jjinux'

bugs.python.org fields:

activity = <Date 2017-09-27.21:43:24.307>
actor = 'eric.araujo'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2007-02-01.02:20:07.000>
creator = 'jjinux'
dependencies = []
files = ['8287']
hgrepos = []
issue_num = 1649329
keywords = ['patch']
message_count = 22.0
messages = ['54994', '54995', '54996', '54997', '54998', '54999', '55000', '55001', '55002', '84641', '119392', '120531', '122390', '122393', '122394', '122395', '122953', '122964', '128692', '128693', '303184', '303185']
nosy_count = 7.0
nosy_names = ['loewis', 'gregory.p.smith', 'ajaksu2', 'jjinux', 'tarek', 'chrysn', 'eric.araujo']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue1649329'
versions = ['Python 3.7']

[jjinux]

If you distribute your code in a zipped egg, you can't use translation catalogs stored in that egg. That's because gettext.py only knows how to find the translation catalogs in the actual filesystem.

This wouldn't be so bad if the "find" function didn't serve double duty. On the one hand, it implements the "find" algorithm to pick the right languages and fallbacks. On the other hand, it actually resolves the files in the filesystem. It would be nice if these were separate. It seems like people in projects like Pylons are stuck copying code from the find function in order to work around this problem.

Perhaps gettext can be updated to know about eggs. Perhaps setting localedir to something like "egg://..." would enable this functionality.

[loewis]

I fail to see the bug. gettext.find behaves as specified; if you want something else, don't use that function.

If you want to load a .mo file from a zip file, you should be able to create a GNUTranslation object directly, from a file-like object.

I don't think that gettext should support eggs directly. If you want it to do things other than loading from file systems, you should generalize that appropriately. One appropriate generalization could be the introduction of a directory-like object, where you can do .exists(relpath), and .open(relpath). However, introduction of directory-like objets is PEP material.

Reclassifying this as a feature request.

[jjinux]

Sorry, yes, this is a feature request. The most important thing that I'm requesting is to refactor the code. That "find" function implements a certain algorithm that has nothing to do with the filesystem.

[loewis]

Do you have a proposal on how to do the refactoring? It would be fine to split find into two parts; it would not be acceptable (to me) to teach it egg: URLs.

[jjinux]

File Added: gettext.patch

[jjinux]

Ok, I've uploaded a patch with a possible way to refactor the code. I'm okay if you mark this bug as WONTFIX. I'm raising it as a concern because I know there are a lot of people out there using Web frameworks such as Pylons, Turbo Gears, etc. that are going to have to duplicate the code in "find" in order to use eggs. I hate to have see them do that. :-/

[loewis]

Ok, I see how this fixes find(). What about translation() (which calls find, then also performs open())? Wouldn't this need to be fixed as well?

Also, what about os.path.join? It might depend on the structure of the storage, as well (suppose you have the .mo data stored in a relational database)?

In principle, I'm willing to fix this. I just want to avoid adding a hack.

For example, it might be sensible to come up with a MOOpener interface, that has an .exists method, taking a list of strings, a .join method, taking a list of strings and returning something that can be passed to the .open method, which would take something that .join returned and returning a file-like object. It might be possible to extend the meaning of the localedir parameter to be either a string (understood as a directory name), or a MOOpener object.

In any case, people *will* have to duplicate find for the moment, as any changes we discuss will only appear in 2.6.

[jjinux]

What about translation() (which calls find, then also performs open())? Wouldn't this need to be fixed as well?

Perhaps, but it's probably not as critical. I.e. if the code gets duplicated to deal with eggs, it's not as big a loss. I really wish the distutils guys would weigh in on this one.

Also, what about os.path.join?

I think it's relatively harmless. One can work around that. One can't work around os.path.exists.

In principle, I'm willing to fix this. I just want to avoid adding a hack.

I agree. The patch isn't smelling too pretty. Neither of us wants to commit to a virtual filesystem layer which is how I've handled problems like this in the past. However, doing nothing results in a ton of code being duplicated in any project that uses gettext and zipped eggs. Should we see if Phillip Eby has any ideas since he's Mr. Eggs?

For example, it might be sensible to come up with a MOOpener interface,
that has an .exists method, taking a list of strings, a .join method,
taking a list of strings and returning something that can be passed to the
.open method, which would take something that .join returned and returning
a file-like object. It might be possible to extend the meaning of the
localedir parameter to be either a string (understood as a directory name),
or a MOOpener object.

That's a light virtual filesystem layer ;) If you want that, I have code for that.

In any case, people *will* have to duplicate find for the moment, as any changes we discuss will only appear in 2.6.

True. Alas, this may be a moot point.

Thanks for your time.

[jjinux]

Perhaps instead of committing to a special virtual filesystem layer, perhaps we should just take something that matches some subset of the posix interface. We can document the functions we need.

[devdanzin]

Not really a distutils issue, but fits the current distribution
discussion scope.