colbert qps
0
20
40
60
80
throughput, qps
quantization config, colbert rescorer (hybrid prefetch, k=20, pf=100, concurrency=8)
47
no-quant
(f32)
44
no-quant
(f16)
45
no-quant
(turbo4)
44
scalar
int8
42
product
x16
43
binary
(f32)
45
binary
(f16)
41
turbo4
tq1