BUG: UndefinedVariableError in DataFrame.query if name is type

This issue has been tracked since 2022-09-21.

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

df = pd.DataFrame({'x': [1, 2, 3], 'y': [4, 5, 6]})

class A:
    a = 1

df.query('x == @A.a')  # <-- this produces the error

Issue Description

In Pandas 1.5.0 it is not possible to use class as name in DataFrame.query:

>>> import pandas as pd
>>> 
>>> df = pd.DataFrame({'x': [1, 2, 3], 'y': [4, 5, 6]})
>>> 
>>> class A:
...     a = 1
... 
>>> df.query('x == @A.a')  # <-- this produces the error
Traceback (most recent call last):
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/scope.py", line 198, in resolve
    return self.resolvers[key]
  File "/usr/lib64/python3.10/collections/__init__.py", line 986, in __getitem__
    return self.__missing__(key)            # support subclasses that define __missing__
  File "/usr/lib64/python3.10/collections/__init__.py", line 978, in __missing__
    raise KeyError(key)
KeyError: 'A'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/scope.py", line 209, in resolve
    return self.temps[key]
KeyError: 'A'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/util/_decorators.py", line 317, in wrapper
    return func(*args, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/frame.py", line 4464, in query
    res = self.eval(expr, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/util/_decorators.py", line 317, in wrapper
    return func(*args, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/frame.py", line 4602, in eval
    return _eval(expr, inplace=inplace, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/eval.py", line 354, in eval
    parsed_expr = Expr(expr, engine=engine, parser=parser, env=env)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 813, in __init__
    self.terms = self.parse()
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 832, in parse
    return self._visitor.visit(self.expr)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 415, in visit
    return visitor(node, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 421, in visit_Module
    return self.visit(expr, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 415, in visit
    return visitor(node, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 424, in visit_Expr
    return self.visit(node.value, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 415, in visit
    return visitor(node, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 723, in visit_Compare
    return self.visit(binop)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 415, in visit
    return visitor(node, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 536, in visit_BinOp
    op, op_class, left, right = self._maybe_transform_eq_ne(node)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 458, in _maybe_transform_eq_ne
    right = self.visit(node.right, side="right")
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 415, in visit
    return visitor(node, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 645, in visit_Attribute
    resolved = self.visit(value).value
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 415, in visit
    return visitor(node, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 549, in visit_Name
    return self.term_type(node.id, self.env, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/ops.py", line 85, in __init__
    self._value = self._resolve_name()
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/ops.py", line 109, in _resolve_name
    res = self.env.resolve(local_name, is_local=is_local)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/scope.py", line 211, in resolve
    raise UndefinedVariableError(key, is_local) from err
pandas.errors.UndefinedVariableError: name 'A' is not defined

I think it is after this commit:
8b36142

I can get expected behavior if I comment this condition:
https://github.com/pandas-dev/pandas/blob/main/pandas/core/computation/ops.py#L105

Expected Behavior

>>> import pandas as pd
>>> 
>>> df = pd.DataFrame({'x': [1, 2, 3], 'y': [4, 5, 6]})
>>> 
>>> class A:
...     a = 1
... 
>>> df.query('x == @A.a')  # <-- this produces the error
   x  y
0  1  4

Installed Versions

INSTALLED VERSIONS

commit : 87cfe4e
python : 3.10.6.final.0
python-bits : 64
OS : Linux
OS-release : 5.19.9-200.fc36.x86_64
Version : #1 SMP PREEMPT_DYNAMIC Thu Sep 15 09:49:52 UTC 2022
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.5.0
numpy : 1.23.3
pytz : 2022.2.1
dateutil : 2.8.2
setuptools : 65.3.0
pip : 22.2.2
Cython : None
pytest : 7.1.3
hypothesis : None
sphinx : 5.1.1
blosc : None
feather : None
xlsxwriter : 3.0.3
lxml.etree : 4.9.1
html5lib : 1.1
pymysql : None
psycopg2 : 2.9.3
jinja2 : 3.1.2
IPython : 8.5.0
pandas_datareader: None
bs4 : 4.11.1
bottleneck : 1.3.5
brotli : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : 3.6.0
numba : None
numexpr : 2.8.3
odfpy : None
openpyxl : 3.0.10
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.9.1
snappy : None
sqlalchemy : 1.4.41
tables : None
tabulate : 0.8.10
xarray : None
xlrd : 2.0.1
xlwt : None
zstandard : None
tzdata : None

phofl wrote this answer on 2022-09-21

Can confirm

commit 8b36142a0e7c1071e9fd9ff37f579ebdef056838
Author: Khor Chean Wei <[email protected]>
Date:   Fri Aug 12 08:15:48 2022 +0800

    Error raised message: DataFrame eval will not work with 'Timestamp' column name (#47273)

cc @weikhor

rhshadrach wrote this answer on 2022-09-21

Somewhat tangential, but with f-strings, is there any advantage of using @? E.g. in this case

df.query(f'x == {A.a}')

This has the added benefit of linters detecting the usage of A.a.

More Details About Repo
Owner Name pandas-dev
Repo Name pandas
Full Name pandas-dev/pandas
Language Python
Created Date 2010-08-24
Updated Date 2022-10-04
Star Count 35430
Watcher Count 1120
Fork Count 15089
Issue Count 3589

YOU MAY BE INTERESTED

Issue Title Created Date Updated Date