AttributeError: module 'intel_extension_for_pytorch' has no attribute 'QuantConf' ipex.__version__

[vgoklani]

Hello - I'm running ipex: '1.10.0+cpu'

and torch version: '1.10.0+cu113'

when running this:

conf = ipex.QuantConf(dtype=torch.int8) 

I'm getting this error:

AttributeError: module 'intel_extension_for_pytorch' has no attribute 'QuantConf'

Please help!

Also, what sort of performance gain should I expect when using int8 quantization with ipex vs the standard qint8 quantization straight from pytorch:

torch.quantization.quantize_dynamic(model=self._model, qconfig_spec={torch.nn.Linear, torch.nn.Embedding}, dtype=torch.qint8)

thanks!

[jgong5]

There is a bug in the doc page. We will update it soon. Please use ipex.quantization.QuantConf instead.

Also, what sort of performance gain should I expect when using int8 quantization with ipex vs the standard qint8 quantization straight from pytorch

You listed an example of dynamic quantization with native pytorch. IPEX only supports static quantization. In general, static quantization avoids most runtime quant/dequant cost and is definitively faster than dynamic quantization. If you compare the perf of static quantization between the native pytorch and IPEX, IPEX brings more int8 fusion supports like conv+add+relu and it also supports mixed input/output data types, e.g., quantized input and fp32 output to avoid further quant/dequant overhead at the output. I cannot tell the numbers of perf gain since it is model by model.

[vgoklani]

thanks - could you please provide updated example code for how to do the int8 calibration for BERT?

It appears that the dtype argument here is no longer valid:

conf = ipex.quantization.QuantConf(dtype=torch.int8) 

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-13-85126aa65212> in <module>
----> 1 conf = ipex.quantization.QuantConf(dtype=torch.int8)

TypeError: __init__() got an unexpected keyword argument 'dtype'

[jgong5]

@jingxu10 Could you please help?

[jingxu10]

Hi @vgoklani , could you try the following code? We will update it to the website soon.

import torch

model = Model()
model.eval()
data = torch.rand(<shape>)

import intel_extension_for_pytorch as ipex
conf = ipex.quantization.QuantConf(qscheme=torch.per_tensor_affine) 

for d in calibration_data_loader(): 
  # conf will be updated with observed statistics during calibrating with the dataset 
  with ipex.quantization.calibrate(conf):
    model(d) 

conf.save('int8_conf.json', default_recipe=True)
with torch.no_grad():
  model = ipex.quantization.convert(model, conf, torch.rand(<shape>)) 
  model.save('quantization_model.pt')

[EikanWang]

@vgoklani , I guess you have resolved this issue by following Jing's sample. I will close this issue first and pls. feel free to reopen it if you still encounter this issue. Thanks.

[arjun0502]

I am using IPEX v1.12.100 and encountered the following error: module 'intel_extension_for_pytorch.quantization' has no attribute 'QuantConf'. @jgong5 @EikanWang How can I resolve this?

The line causing this error is the following: conf = ipex.quantization.QuantConf(args.configure_dir)

[XiaobingSuper]

@arjun0502 , for 1.12, we introduce a new quantization API, you can follow the https://github.com/intel/intel-extension-for-pytorch/blob/master/docs/tutorials/features/int8.md to use the new quantization API, thanks!