Skip to content

fix(input_output): keyword-only pipeline parameters crash process_args with TypeError - #1785

Open
Anai-Guo wants to merge 1 commit into
modelscope:masterfrom
Anai-Guo:fix-process-args-kwonly
Open

Anai-Guo wants to merge 1 commit into
modelscope:masterfrom
Anai-Guo:fix-process-args-kwonly

Conversation

@Anai-Guo

Copy link
Copy Markdown

The bug

process_arg_type_annotation takes two required parameters:

def process_arg_type_annotation(arg, default_value):   # input_output.py:241

process_args has four call sites. The two positional-argument branches pass both parameters; the two keyword-only branches pass only one:

line branch call
297 positional, no default process_arg_type_annotation(arg, None) ✅
304 positional, with default process_arg_type_annotation(arg, value) ✅
311 keyword-only, no default process_arg_type_annotation(kwarg) ❌
316 keyword-only, with default process_arg_type_annotation(kwarg) ❌

So any pipeline whose signature contains a keyword-only parameter fails schema generation with:

TypeError: process_arg_type_annotation() missing 1 required positional argument: 'default_value'

The second half of the same bug

Fixing only the arity is not enough. ast keeps a None placeholder in kw_defaults for every keyword-only arg declared without a default, so len(args.kw_defaults) == len(args.kwonlyargs) always holds. That makes n_kwargs - n_kwargs_default always 0, so:

  • the line-311 loop iterates over an empty slice and never runs — every keyword-only arg, with or without a default, goes through the line-316 loop;
  • that loop then unconditionally does dft.value and appends has_default=True, which would AttributeError on the None placeholder and mislabel a required kw-only arg as optional.

So the fix reads the default through convert_to_value — exactly what the positional branch at line 304 already does — and derives has_default from whether a default is actually present.

Verification

Ran against the real modelscope.utils.input_output (no stubs), before and after:

                                  before                                            after
positional a:int, b:str='x'       [('a','int',False,None), ('b','str',True,'x')]     unchanged
kw-only  *, mode:str='fast'       TypeError (see above)                              [..., ('mode','str',True,'fast')]
kw-only  *, mode:str              TypeError (see above)                              [..., ('mode','str',False,None)]

Three tests added as ProcessArgsTest in tests/utils/test_input_output.py. The two keyword-only tests error on master with the exact TypeError; the positional test passes both before and after, pinning that the existing path is untouched:

# master
ERROR: test_keyword_only_arg_with_default    TypeError: ... missing 1 required positional argument: 'default_value'
ERROR: test_keyword_only_arg_without_default TypeError: ... missing 1 required positional argument: 'default_value'
Ran 3 tests — FAILED (errors=2)

# with this patch
Ran 3 tests in 0.001s — OK

yapf==0.30.0 (the pinned pre-commit rev) reports no diff on either file, isort is clean, and flake8 with the setup.cfg select/ignore set reports nothing new.

Out of scope, but noticed while verifying

convert_to_value ends in return UnhandledKeyType(), and UnhandledKeyType is not defined anywhere in the repo — so a non-literal default (e.g. def f(self, a: list = [1, 2])) raises NameError: name 'UnhandledKeyType' is not defined. That is pre-existing on the positional path and reproduces on master without this patch, so I left it alone rather than guessing at the intended sentinel type. Happy to send a follow-up if you tell me what it should return.

🤖 Generated with Claude Code

…rd-only args

process_arg_type_annotation takes (arg, default_value), and the two positional
branches of process_args pass it. The two keyword-only branches called it with
only the arg, so any pipeline signature containing a keyword-only parameter
raised TypeError instead of producing its schema.

Also guard the default itself: ast keeps a None placeholder in kw_defaults for
keyword-only args declared without a default, so read it through
convert_to_value (as the positional branch does) and report has_default
accordingly instead of unconditionally True + dft.value.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant