Allow mappings as globals (was: Fix dictionary subclass ...)

@loewis

BPO	1402289
Nosy	@loewis, @arigo, @rhettinger, @terryjreedy, @methane, @pppery
Files	dictsubclassexec.patch: patch file dictsubclassexec.42135.patch: new version of patch file mappingglobals.42206.patch: updated patch for mappings as globals

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-04-15.00:52:41.372>
created_at = <Date 2006-01-11.00:24:45.000>
labels = ['interpreter-core', 'type-feature', '3.8']
title = 'Allow mappings as globals (was: Fix dictionary subclass ...)'
updated_at = <Date 2019-04-15.00:52:41.370>
user = 'https://bugs.python.org/crutchergmail'

bugs.python.org fields:

activity = <Date 2019-04-15.00:52:41.370>
actor = 'rhettinger'
assignee = 'none'
closed = True
closed_date = <Date 2019-04-15.00:52:41.372>
closer = 'rhettinger'
components = ['Interpreter Core']
creation = <Date 2006-01-11.00:24:45.000>
creator = 'crutcher_gmail'
dependencies = []
files = ['6946', '6947', '6948']
hgrepos = []
issue_num = 1402289
keywords = []
message_count = 26.0
messages = ['49288', '49289', '49290', '49291', '49292', '49293', '49294', '49295', '49296', '49297', '49298', '49299', '49300', '49301', '49302', '49303', '49304', '49305', '49306', '49307', '78882', '113446', '340180', '340186', '340237', '340239']
nosy_count = 7.0
nosy_names = ['loewis', 'arigo', 'rhettinger', 'terry.reedy', 'crutcher_gmail', 'methane', 'ppperry']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue1402289'
versions = ['Python 3.8']

There is an inconsistancy in the way that dictionary
subclasses behave
when they are used as as namespaces in execs.

Basically, while python 2.4 permits the usage of
dictionary subclasses
for local environments, it still bypasses the subclass
functions and
uses the C API for global environments. The attached
patch (and
unittest!) addresses this issue.

I'm pretty sure we keep the fast path in this.

Logged In: YES
user_id=80475

Do you have any use cases? AFAICT, there were no unmet
needs with locals option.

Also, there was a reason that globals weren't included but I
remember what it was.

Logged In: YES
user_id=1424288

With the ability to use dictionary subclasses for local and
global environments, it becomes very easy to implement
execution environments with extended variable semantics for
special purposes.

This includes driving interactive processes, config file
processors, type checking, lazy object construction, and
read-only variables.

As it currently stands, the semantics are inconsistent. The
local case should not have been changed if the global case
was not going to be changed.

Logged In: YES
user_id=1424288

Here's an example of a little typed environment. It's not
the most robust, but it gets you thinking.

import code
class TypedDictionary(dict):
 def __setitem__(self, key, value):
   if self.has_key(key):
     t = type(self[key])
     if t != type(value):
       try:
         value = t(value)
       except Exception:
         raise TypeError, \
             "illegal assignment to '%s':" \
             " %s cannot be coerced to %s" \
               % (key, type(value), t)
   dict.__setitem__(self, key, value)
code.interact(local=TypedDictionary())

Logged In: YES
user_id=1424288

Here's a more interesting example. This works fine, unless
the variables fall through to the global dictionary, or some
code marks them global.

import code

class ManagedVariable:
  def get(self):
    return None

  def set(self, value):
    pass

  def delete(self):
    # Return false to stop the delete.
    return True

class ManagedEnvironment(dict):
 def __setitem__(self, key, value):
   if self.has_key(key):
     if isinstance(dict.__getitem__(self, key),
ManagedVariable):
       dict.__getitem__(self, key).set(value)
       return
   dict.__setitem__(self, key, value)

 def __getitem__(self, key):
   if self.has_key(key):
     if isinstance(dict.__getitem__(self, key),
ManagedVariable):
       return dict.__getitem__(self, key).get()
   return dict.__getitem__(self, key)

 def __delitem__(self, key):
   if self.has_key(key):
     if isinstance(dict.__getitem__(self, key),
ManagedVariable):
       if not dict.__getitem__(self, key).delete():
         return
   dict.__delitem__(self, key)


class RangedInt(ManagedVariable):
  def __init__(self, value, (low, high)):
    self.value = value
    self.low = low
    self.high = high

  def get(self):
    return self.value

  def set(self, value):
    if value < self.low:
      value = self.low
    if value > self.high:
      value = self.high
    self.value = value


class FunctionValue(ManagedVariable):
  def __init__(self, get_func = None, set_func = None,
del_func = None):
    self.get_func = get_func
    self.set_func = set_func
    self.del_func = del_func

  def get(self):
    if self.get_func:
      return self.get_func()
    return None

  def set(self, value):
    if self.set_func:
      self.set_func(value)

  def delete(self):
    if self.del_func:
      return self.del_func()
    return True

class Constant(ManagedVariable):
  def __init__(self, value):
    self.value = value

  def get(self):
    return self.value

  def delete(self):
    return False


import time
d = ManagedEnvironment()
d['ranged'] = RangedInt(1, (0, 100))
d['time'] = FunctionValue(lambda: time.time())
d['constant'] = Constant(42)

code.interact(local=d)

Logged In: YES
user_id=80475

Okay, I will take it under consideration. Also, I'll try to
remember the reason it wasn't done the first time around.

No promises though -- by itself consistency is a weak
motivation.

Logged In: YES
user_id=21627

I think the history is this: it originates from
python.org/sf/215126, which requested that you can pass dict
subtypes to eval. Armin noted that eval will always produce
LOAD/STORE_NAME, so just modifying these opcodes is
sufficient to fulfil the feature request.

Logged In: YES
user_id=1424288

Well, why fix it for eval but not for exec? I don't think
the time hit is noticeable, I ran 'time make test' twice
each on the trunk with and without the patch. Here are the
results:

Trunk:
real 9m17.117s
user 3m30.930s
sys 0m35.417s

real 9m9.471s
user 3m31.484s
sys 0m34.978s

Patch:
real 9m32.469s
user 3m40.134s
sys 0m36.140s

real 9m6.779s
user 3m27.529s
sys 0m34.716s

Logged In: YES
user_id=4771

Provided this can be done with no measurable performance hit,
I guess that I'm fine with the idea. The patch needs a bit
more work, though: I don't see why it should accept dict
subclasses as globals but not arbitrary mappings (as it now
does for the locals). This is mainly an issue of removing
a few checks in various places, like EXEC_STMT and the eval()
and execfile() built-ins.

There is a missing exception check/clear in the part about
LOAD_NAME, after the PyObject_GetItem(f->f_globals, w).

A side note: in the current trunk already, LOAD_GLOBAL
contains a couple of checks, namely PyString_CheckExact()
and hash != -1. We might be able to prove in advance that
these two conditions are always true. We could then remove
the checks. Not sure the difference measurable, though.

Logged In: YES
user_id=1424288

I'm going to fix the missed exception test this weekend, and
try to get an updated patch to you.

Logged In: YES
user_id=1424288

I've fixed up the exception case, and extended the test case
to check for it. Is there anything else I can do to get this in?

Haven't had a chance to work on this one. Unassigning.

I believe this is covered by the PEP-3003 3.2 change moratorium.

13 years past from this proposed. Is this still good feature for Python?

Personally, I dislike adding more dynamic flexibility to Python name space.

Python is very dynamic language, and it made difficult to make Python faster.
For example, PHP is not so dynamic as Python, and it is one of reasons why PHP VM is now much faster than Python VM. They can easily do more optimization like function inlining statically.

I'm interested in optimizing Python. But I'm not interested in adding more "slow path" we need to maintain forever.

I am willing to close it.

Navigation Menu

Repository navigation

Allow mappings as globals (was: Fix dictionary subclass ...) #42784

9 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Allow mappings as globals (was: Fix dictionary subclass ...) #42784

Description

Activity

crutchergmail commented on Jan 11, 2006

rhettinger commented on Jan 11, 2006

crutchergmail commented on Jan 11, 2006

crutchergmail commented on Jan 11, 2006

crutchergmail commented on Jan 11, 2006

rhettinger commented on Jan 12, 2006

loewis commented on Jan 12, 2006

crutchergmail commented on Jan 12, 2006

arigo commented on Jan 14, 2006

crutchergmail commented on Jan 22, 2006

crutchergmail commented on Jan 22, 2006

9 remaining items

rhettinger commented on Jan 2, 2009

terryjreedy commented on Aug 9, 2010

methane commented on Apr 14, 2019

terryjreedy commented on Apr 14, 2019

pppery commented on Apr 14, 2019

rhettinger commented on Apr 15, 2019

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions