Searched refs:quantized (Results 1 – 5 of 5) sorted by relevance
76 float quantized = position * STEP_SIZE; in getInterpolation() local77 float difference = input - quantized; in getInterpolation()
5027 * A version of quantized LSTM, using 16 bit quantization for internal5036 * cell. Tensor is quantized with a fixed quantization range of5089 * inside the LSTM cell. Bias is quantized with scale being a product5094 * inside the LSTM cell. Bias is quantized with scale being a product5099 * inside the LSTM cell. Bias is quantized with scale being a product5104 * inside the LSTM cell. Bias is quantized with scale being a product5108 * previous time step of the LSTM cell. It is quantized using a5113 * cell from previous time-step. Tensor is quantized with a fixed5121 * the current time step. Tensor is quantized using a quantization5126 * Tensor is quantized with a fixed quantization range of [-1, 127/128][all …]
26 # i1 is a quantized input of shape (2, 256, 256, 3), with scale = 0.5, zero point = 128