• Ask a Question
  • Create a Poll
150
    Ask a Question
    Cancel
    150
    More answer You can create 5 answer(s).
      Ask a Poll
      Cancel

      Why isn't fromfile-prefix-chars in Python argparse working?

      I’m trying to use the fromfile-prefix-chars feature of argparse in Python to load all my command line arguments from a file, but it keeps complaining that I haven’t specified some argument.

      The code:

      import argparse  def go():    parser = argparse.ArgumentParser(fromfile_prefix_chars='@')    parser.add_argument("--option1")    parser.add_argument("--option2", type=int, required=True)    args = parser.parse_args()  if __name__ == "__main__":     go() 

      The argument file:

      --option1 foo --option2 1234 

      The command line and output:

      $ python testargparse.py @testargs usage: testargparse.py [-h] [--option1 OPTION1] --option2 OPTION2 testargparse.py: error: argument --option2 is required 

      You can see that I’m providing the required argument in the file, but argparse isn’t seeing it.

      Asked by Barrettlucianojoan on July 16, 2020 in Python.
      4 Answers

      From the documentation:

      Arguments read from a file must by default be one per line … and are treated as if they were in the same place as the original file referencing argument on the command line. So in the example above, the expression [‘-f’, ‘foo’, ‘@args.txt’] is considered equivalent to the expression [‘-f’, ‘foo’, ‘-f’, ‘bar’].

      In the example:

      fp.write('-f\nbar') 

      So the file contains:

      -f bar 

      In other words, each of the file lines corresponds to one ‘word’ (blank separated) in the command line. --option1=foo is one word. --option1 foo is interpreted just as though it was quoted in the command line,eg. prog.py '--option1 foo' '--option2 1234'

      The https://docs.python.org/dev/library/argparse.html#argparse.ArgumentParser.convert_arg_line_to_args has a custom function that will split lines on spaces. Experiment with that if you want to stick with the argument file.


      import argparse  with open('args.txt', 'w') as fp:     fp.write('--option1 foo\n--option2 1234')  # error     # but works with modifed 'convert...'     #fp.write('--option1=foo\n--option2=1234')  # works     #fp.write('--option1\nfoo\n--option2\n1234') # works  def convert_arg_line_to_args(arg_line):     for arg in arg_line.split():         if not arg.strip():             continue         yield arg """ default line converter: def convert_arg_line_to_args(self, arg_line):     return [arg_line] """  def go():    parser = argparse.ArgumentParser(fromfile_prefix_chars='@')    parser.convert_arg_line_to_args = convert_arg_line_to_args    parser.add_argument("--option1")    parser.add_argument("--option2", type=int, required=True)    args = parser.parse_args(['@args.txt'])    print args  if __name__ == "__main__":     go() 

      @toes suggested using shlex to parse the file. shlex has a nice feature in that it strips of unnecessary quotes.

      shlex can be used to split individual lines of the file.

      def sh_split(arg_line):     for arg in shlex.split(arg_line):         yield arg parser.convert_arg_line_to_args = sh_split 

      Or it can replace the whole @file read method (_read_args_from_files)- this should function the same as @toes answer, except that the @file string can be anywhere in the commandline (or even be repeated).

      def at_read_fn(arg_strings):     # expand arguments referencing files     new_arg_strings = []     for arg_string in arg_strings:         if not arg_string or not arg_string.startswith('@'):             new_arg_strings.append(arg_string)         else:             with open(arg_string[1:]) as args_file:                 arg_strings = shlex.split(args_file.read())                 new_arg_strings.extend(arg_strings)     return new_arg_strings parser._read_args_from_files = at_read_fn 

      Obviously a cleaner production version would modify these methods in an ArgumentParser subclass.

      Answered by Kendrickkaseylakeisha on July 16, 2020..

      The problem is that, when specified in a file, each argument must have an ‘=’ between it and the option name. While argparse is somewhat more flexible on that format when run from the command line (where space or = is ok), when run from the file it must have an ‘=’.

      So, a working argument file would be:

      --option1=foo --option2=1234 

      Something else to be aware of, be sure you don’t have any extra whitespace at the end of the lines or that whitespace will get included with the option when argparse reads the file.

      Answered by Judsonhollybernadine on July 16, 2020..

      I think there’s a better answer to this: use shlex.

      if sys.argv[1].startswith('@'):     args = parser.parse_args( shlex.split(open(sys.argv[1][1:]).read()) ) else:     args = parser.parse_args() 

      This allows you to specify args in a file in a more natural way e.g., it allows using spaces or equals sign to specify your args on a single line as in:

      arg1 arg2 --opt1 'foo' --opt2='bar' 

      shlex.split splits this as you would expect:

      ['arg1', 'arg2', '--opt1', 'foo', '--opt2=bar'] 

      The only thing this method doesn’t have is that it expects the @file.txt to be the first argument.

      Answered by John on July 16, 2020..

      Try to this way

      # encoding: utf-8 import imp import argparse   class LoadConfigAction(argparse._StoreAction):     NIL = object()      def __init__(self, option_strings, dest, **kwargs):         super(self.__class__, self).__init__(option_strings, dest)         self.help = "Load configuration from file"      def __call__(self, parser, namespace, values, option_string=None):         super(LoadConfigAction, self).__call__(parser, namespace, values, option_string)          config = imp.load_source('config', values)          for key in (set(map(lambda x: x.dest, parser._actions)) & set(dir(config))):             setattr(namespace, key, getattr(config, key)) 

      Usage

      parser.add_argument("-C", "--config", action=LoadConfigAction, default=None) parser.add_argument("-H", "--host") 

      Example config (real is python file)

      # Config example /etc/my.conf import os  # Parameter definition host = os.getenv("HOST", "127.0.0.1") 
      Answered by Jonsallieyolanda on July 16, 2020..