argparse.add_mutually_exclusive_group fails for optional positional arguments #86020

Activity

1. rrt commented on Sep 24, 2020

   

   rrtmannequin

   on Sep 24, 2020

   MannequinAuthor

   More actions

   The following code:

   group = parser.add_mutually_exclusive_group()
   group.add_argument('--install-only', action='store_true',
                       help='just install the program, do not run it')
   group.add_argument('args', metavar='ARGUMENT', nargs='*', default=None,
                       help='arguments to PROGRAM')

   gives the following error:

       group.add_argument('args', metavar='ARGUMENT', nargs='*',
     File "/usr/lib/python3.8/argparse.py", line 1398, in add_argument
       return self._add_action(action)
     File "/usr/lib/python3.8/argparse.py", line 1621, in _add_action
       raise ValueError(msg)
   ValueError: mutually exclusive arguments must be optional

   But the 'args' argument *is* optional, as there can be 0 of them.

   👍React with 👍1Reacted by Glenn 'devalias' Grant
2. rrt

   added

   3.8 (EOL)end of lifeend of life

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   on Sep 24, 2020
3. paulj3 commented on Oct 17, 2020

   

   paulj3mannequin

   on Oct 17, 2020

   Mannequin

   More actions

   Give the positional a non-none default:

   e.g.

       group.add_argument('args', metavar='ARGUMENT', nargs='*', default=[],
                        help='arguments to PROGRAM')

   Since a '*' or '?' positional works with an empty list of arguments, it is "always seen". It requires some special handling to allow it to work in the mutually exclusive context. It's a tricky piece of code that might not be well documented (if at all).
4. rrt commented on Oct 17, 2020

   

   rrtmannequin

   on Oct 17, 2020

   MannequinAuthor

   More actions

   Thanks for the hint; could this be documented, please?
5. ezio-melotti

   transferred this issue fromon Apr 10, 2022
6. allisonkarlitskaya commented on May 2, 2022

   

   allisonkarlitskaya

   on May 2, 2022

   Contributor

   More actions

   There's another issue here, even with the workaround above: help output for positional parameters appearing as part of mutually exclusive groups is often broken.

   Specifically: when a mutually exclusive group contains positional arguments and the usage line in the --help output is long enough that it needs to be wrapped, the group is no longer displayed as mutually exclusive.

   Here's an example:

   #!/usr/bin/python3
   
   import argparse
   
   parser = argparse.ArgumentParser()
   parser.add_argument('--a-flag')
   parser.add_argument('--b-flag')
   parser.add_argument('--c-flag')
   
   group = parser.add_mutually_exclusive_group()
   group.add_argument('--a-only')
   group.add_argument('--b-only')
   group.add_argument('positional', nargs='*', default=[])
   
   args = parser.parse_args()

   If you ask for --help with a wide enough terminal, you'll see correct output, with the "only" flags marked as mutually exclusive to the positional parameters:

   usage: broken.py [-h] [--a-flag A_FLAG] [--b-flag B_FLAG] [--c-flag C_FLAG] [--a-only A_ONLY | --b-only B_ONLY | positional ...]
   

   but if you make your terminal smaller, you get (incorrect):

   usage: broken.py [-h] [--a-flag A_FLAG] [--b-flag B_FLAG] [--c-flag C_FLAG] [--a-only A_ONLY]
                    [--b-only B_ONLY]
                    [positional ...]
   

   Removing the position parameters (ie: delete the group.add_argument('positional', ...)) gives the correct output. Even in very small terminals (30 columns), it absolutely refuses to ever split the group onto multiple lines:

   usage: broken.py [-h] [--a-flag A_FLAG]
                    [--b-flag B_FLAG]
                    [--c-flag C_FLAG]
                    [--a-only A_ONLY | --b-only B_ONLY]
   

   This seems to be caused by the code in _format_usage() which attempts to consider optionals and positionals together in the unwrapped case:

               # build full usage string
               format = self._format_actions_usage
               action_usage = format(optionals + positionals, groups)
               usage = ' '.join([s for s in [prog, action_usage] if s])
               pos_parts = _re.findall(part_regexp, pos_usage)

   but considers them separately in case the output is too long:

               # wrap the usage parts if it's too long
               text_width = self._width - self._current_indent
               if len(prefix) + len(usage) > text_width:
                   ...
                   opt_usage = format(optionals, groups)
                   pos_usage = format(positionals, groups)

   which is exacerbated by this code in _format_actions_usage() which assumes that the actions will be presented for formatting, complete, and in exactly the same order as they appear in the group:

                   start = actions.index(group._group_actions[0])
                   ...
                   end = start + len(group._group_actions)
                   if actions[start:end] == group._group_actions:

   Indeed, if you insert the positional parameters into group before the flag parameters, the grouping will always be broken, even if the terminal is infinitely wide:

   #!/usr/bin/python3
   
   import argparse
   
   parser = argparse.ArgumentParser()
   group = parser.add_mutually_exclusive_group()
   group.add_argument('positional', nargs='*', default=[])
   group.add_argument('--flag')
   
   args = parser.parse_args()

   produces this incorrect output:

   usage: broken.py [-h] [--flag FLAG] [positional ...]
   

   👍React with 👍1Reacted by ax672d61
7. erlend-aasland

   added this to Argparse issueson May 19, 2022
8. serhiy-storchaka commented on Oct 10, 2024

   

   serhiy-storchaka

   on Oct 10, 2024

   Member

   More actions

   The original issue was fixed (see #72795). Now positional arguments with nargs='*' are always considered optional, independently from the default value.

   The bug reported in #86020 (comment) is unrelated to this. It is either a duplicate of #75949 or closely related.
9. serhiy-storchaka

   closed this as not plannedon Oct 10, 2024
10. github-project-automation

    moved this to Doc issues in Argparse issueson Oct 10, 2024