p50
p99
0
500
1000
1500
2000
latency, ms
quantization config, colbert rescorer (hybrid prefetch, k=20, pf=100)
141
451
no-quant
(f32)
143
455
no-quant
(f16)
143
351
no-quant
(turbo4)
137
752
scalar
int8
210
1446
product
x16
148
508
binary
(f32)
152
609
binary
(f16)
155
305
turbo4
tq1