Skip to content

Commit

Permalink
Fix quant_format argument for 4bit quantizer (#21581)
Browse files Browse the repository at this point in the history
### Description
Original argument accepts Enum QuantFormat.QOperator or QuantFormat.QDQ,
but the default value is QOperator.

Change the argument to str to accept QOperator or QDQ and convert to
QuantFormat after parsing.

### Motivation and Context
Bug fix
  • Loading branch information
fajin-corp authored and prathikr committed Aug 3, 2024
1 parent de81dbe commit 8643127
Showing 1 changed file with 3 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -797,8 +797,8 @@ def parse_args():
parser.add_argument(
"--quant_format",
default="QOperator",
type=QuantFormat,
choices=list(QuantFormat),
type=str,
choices=["QOperator", "QDQ"],
help="QuantFormat {QOperator, QDQ}"
"QOperator format quantizes the model with quantized operators directly."
"QDQ format quantize the model by inserting DeQuantizeLinear before the MatMul.",
Expand All @@ -814,7 +814,7 @@ def parse_args():

input_model_path = args.input_model
output_model_path = args.output_model
quant_format = args.quant_format
quant_format = QuantFormat[args.quant_format]

if os.path.exists(output_model_path):
logger.error(f"file {output_model_path} already exists")
Expand Down

0 comments on commit 8643127

Please sign in to comment.