p50
p99
0
500
1000
1500
2000
latency, ms
quantization config, colbert rescorer (hybrid prefetch, k=20, pf=100)
134
1051
no-quant
(f32)
134
782
no-quant
(f16)
137
275
no-quant
(turbo4)
154
416
scalar
int8
145
374
product
x16
150
697
binary
(f32)
135
1094
binary
(f16)
159
421
turbo4
tq1