[PUBLIC] Voyage AI voyage-context-3 Evaluation Results

	A	B	C	D	E	F	G	H	I	J	K	L	M	N	O	P	Q	R	S	T	U	V	W	X	Y	Z	AA	AB	AC	AD	AE	AF	AG	AH	AI	AJ	AK	AL	AM	AN	AO	AP	AQ	AR	AS	AT	AU	AV	AW	AX	AY	AZ	BA	BB	BC	BD	BE	BF	BG	BH	BI	BJ	BK	BL	BM	BN	BO	BP	BQ	BR	BS	BT	BU	BV	BW	BX	BY	BZ	CA	CB	CC	CD	CE	CF	CG	CH	CI	CJ	CK	CL	CM	CN	CO	CP	CQ	CR	CS	CT	CU	CV
1	Send comments and questions to contact@voyageai.com
2
3
4
5		OVERALL	TECH					WEB					CODE				MEDICAL					CONVERSATION				LAW								FINANCE												LONG-CONTEXT				FRENCH					GERMAN							JAPANESE						KOREAN						SPANISH						OTHER
6	Model	AVERAGE	OneSignal	PyTorch	5G	Cohere	AVERAGE	DoorDash	Health4CA	Movies Summary	Kijiji.ca	AVERAGE	LeetCodePython-rtl	DS-1000-rtl	codechef-cpp_5doc	AVERAGE	Mental Health Consulting	COVID QA	ChatDoctor	Medical Instruction	AVERAGE	Dialog Sum	QA Conv	MeetingBank-transcript	AVERAGE	LegalBench Consumer Contracts QA	LegalBench Corporate Lobbying	AILA casedocs	AILA statutes	LeCaRDv2	LegalQuAD	GerDaLIR	AVERAGE	Trade-the-event	RAG benchmark (Apple-10K-2022)	FinanceBench	TAT-QA-rtl	Indian Financial News	FiQA-Personal-Finance-rtl	Stocks Event	ConvFinQA-rtl	FinQA-rtl	News stock	HC3 Finance	AVERAGE	SummScreenFD	QMSum	WikimQA	AVERAGE	Alpaca French Mixtral	hc3_french_ood_faq_fr_gouv	Paraphrasing French	OpenHermes-2.5 French	AVERAGE	Biology German DHBW	Dialogsum German	German Poems	OASST1 German	OpenOrca German	German Legal Sentences	AVERAGE	fujiki 49k Japanese	Japanese CoNaLa	MIRACL Japanese	MS MARCO Japanese Hard Negatives	Sakura Japanese	AVERAGE	Korean Human Judgements	Korean QA Gen Datasets	Grade School Math Korean	Dialog Summary Korean	Mr. TyDi Korean	AVERAGE	Alpaca Spanish	Emotional Response Spanish Dataset	Function Calling Assistant Spanish Pofi v2	Guanaco Spanish	Spanish Passage Retrieval	AVERAGE	bengali_summ_test	dan_news_summ_test	dutch_policy_qa_test	dutch_squad_test	georgian_faq	greek_civics_qa	hebrew_qa	hungarian_summ	ilpost_test	jpwiki_test	norwegian_snl_summ_test	polish_summ_test	russian_reviews	skquad_test	slovak_summ_test	swedish_swefaq	thai_summ_test	turkish_HistQuAD_test	urdu_summ_test	viet_quad_test	arabic_news

7	OpenAI v3 large	60.49%	44.45%	59.38%	62.08%	50.46%	54.09%	46.75%	60.81%	32.40%	72.75%	53.18%	81.20%	48.43%	14.92%	48.18%	53.55%	50.32%	61.16%	75.64%	60.17%	87.20%	65.56%	54.93%	69.23%	69.35%	84.99%	7.94%	24.35%	10.33%	53.33%	21.23%	38.79%	80.80%	79.70%	44.43%	61.62%	66.31%	57.14%	26.37%	56.98%	51.12%	62.38%	51.77%	58.06%	37.40%	30.31%	41.27%	36.33%	82.71%	54.10%	82.98%	63.60%	70.85%	91.58%	82.37%	35.67%	63.48%	91.92%	66.43%	71.91%	77.21%	75.72%	67.35%	90.91%	87.27%	79.69%	79.70%	60.48%	90.07%	86.06%	73.38%	77.94%	27.93%	85.40%	82.81%	84.51%	39.68%	64.06%	50.11%	80.80%	91.65%	64.59%	13.36%	50.88%	61.97%	57.55%	74.53%	92.98%	81.90%	62.07%	57.68%	75.46%	58.30%	77.36%	64.79%	74.86%	38.12%	71.29%	71.40%
8	Cohere v4	62.17%	46.98%	64.15%	67.39%	54.25%	58.19%	47.67%	53.71%	23.35%	77.97%	50.67%	85.33%	52.66%	18.95%	52.31%	53.59%	53.02%	65.58%	75.29%	61.87%	83.49%	67.62%	56.95%	69.36%	67.52%	82.44%	10.60%	22.25%	12.47%	53.13%	26.50%	39.27%	80.64%	75.93%	41.55%	64.92%	74.00%	56.80%	41.94%	64.95%	60.38%	60.62%	52.77%	61.32%	36.05%	32.72%	38.87%	35.88%	83.20%	56.84%	82.84%	65.47%	72.09%	91.38%	78.93%	41.90%	63.43%	92.65%	73.56%	73.64%	78.37%	81.21%	70.49%	94.51%	89.41%	82.80%	83.21%	62.65%	94.34%	89.50%	81.55%	82.25%	29.57%	85.34%	81.92%	85.00%	37.58%	63.88%	65.14%	72.76%	90.09%	67.27%	29.17%	41.75%	81.21%	44.67%	72.61%	92.32%	78.62%	64.94%	60.71%	77.67%	54.27%	73.32%	72.49%	78.90%	54.31%	79.80%	72.35%
9	voyage-3.5	64.56%	50.37%	67.96%	69.78%	58.92%	61.76%	45.43%	62.80%	22.69%	78.53%	52.36%	86.41%	57.09%	33.34%	58.95%	55.90%	51.32%	64.32%	75.32%	61.71%	89.55%	65.45%	62.66%	72.55%	73.69%	88.42%	9.06%	27.74%	13.14%	54.08%	25.11%	41.61%	81.76%	79.31%	49.51%	71.29%	75.78%	59.02%	61.89%	68.69%	63.16%	63.73%	55.80%	66.36%	40.48%	33.07%	38.97%	37.51%	87.62%	62.35%	81.31%	71.97%	75.81%	91.89%	84.65%	44.96%	65.69%	93.18%	72.31%	75.45%	77.26%	80.15%	70.38%	90.08%	88.95%	81.36%	85.43%	66.81%	94.91%	89.98%	85.17%	84.46%	31.04%	83.78%	83.31%	87.57%	36.97%	64.53%	66.07%	83.23%	86.73%	62.34%	40.83%	46.97%	81.19%	52.94%	74.52%	91.50%	82.31%	63.30%	55.83%	82.16%	63.79%	78.10%	72.73%	84.02%	54.78%	81.88%	73.12%
10	voyage-3-large	66.77%	52.86%	70.16%	70.10%	61.05%	63.54%	48.01%	64.86%	28.03%	76.78%	54.42%	92.19%	58.67%	48.96%	66.61%	59.71%	52.66%	68.54%	78.28%	64.80%	90.83%	66.43%	63.49%	73.58%	75.77%	90.08%	11.60%	28.54%	12.46%	56.72%	26.54%	43.10%	81.20%	79.96%	49.19%	72.42%	71.90%	62.10%	52.30%	70.11%	64.94%	64.80%	60.09%	66.27%	40.66%	31.88%	41.65%	38.06%	89.53%	65.10%	83.45%	75.41%	78.37%	93.07%	86.64%	48.46%	68.97%	93.91%	73.81%	77.48%	82.18%	85.14%	73.55%	93.38%	95.04%	85.86%	86.75%	66.61%	96.00%	92.62%	86.53%	85.70%	30.69%	86.30%	83.68%	89.14%	37.20%	65.40%	68.58%	83.10%	89.05%	64.95%	38.12%	54.84%	83.27%	57.49%	75.86%	94.22%	83.31%	62.88%	61.87%	79.37%	63.82%	80.88%	74.78%	88.29%	57.16%	84.78%	76.30%
11	Jina v3 Late Chunking	51.07%	40.14%	54.26%	67.49%	56.27%	54.54%	10.80%	42.05%	6.48%	45.51%	26.21%	90.47%	41.56%	11.51%	47.85%	32.15%	12.53%	44.85%	59.37%	37.22%	70.96%	23.63%	60.15%	51.58%	58.48%	85.02%	8.45%	10.79%	13.05%	30.63%	16.71%	31.88%	67.43%	73.66%	46.93%	33.42%	68.84%	32.18%	14.80%	23.23%	21.77%	48.57%	24.56%	41.40%	32.05%	10.90%	16.08%	19.68%	83.93%	46.16%	82.98%	69.61%	70.67%	95.63%	79.87%	46.24%	55.00%	85.14%	58.36%	70.04%	64.51%	61.13%	29.63%	78.96%	77.10%	62.26%	87.68%	77.42%	95.55%	83.69%	66.06%	82.08%	30.07%	76.13%	63.82%	79.40%	18.48%	53.58%	74.53%	82.78%	81.10%	53.58%	45.93%	67.39%	54.14%	32.01%	78.43%	90.40%	79.67%	62.15%	60.51%	60.14%	59.40%	69.99%	78.48%	57.19%	61.60%	64.20%	73.27%
12	Contextual Retrieval (voyage-3-large)	67.97%	58.79%	74.82%	77.76%	65.76%	69.28%	51.08%	71.37%	28.01%	76.58%	56.76%	94.17%	59.75%	47.62%	67.18%	58.90%	51.44%	68.07%	79.01%	64.35%	92.14%	66.37%	66.80%	75.11%	75.66%	90.73%	11.74%	30.65%	13.26%	54.89%	27.02%	43.42%	83.02%	86.30%	74.40%	74.95%	73.06%	61.96%	50.18%	78.13%	74.41%	67.41%	58.51%	71.12%	42.67%	36.66%	35.24%	38.19%	89.85%	64.41%	78.37%	77.79%	77.60%	93.61%	89.50%	51.22%	68.57%	93.49%	72.30%	78.11%	82.34%	85.22%	73.84%	93.16%	94.14%	85.74%	86.80%	70.00%	98.24%	93.51%	84.45%	86.60%	30.80%	85.50%	81.07%	88.68%	39.27%	65.06%	78.29%	81.90%	88.08%	65.78%	44.08%	63.54%	82.48%	60.84%	77.06%	94.72%	82.94%	53.09%	62.66%	79.83%	65.76%	79.39%	79.29%	88.11%	62.31%	83.74%	77.78%
13	voyage-context-3	74.73%	71.01%	83.62%	84.46%	77.40%	79.12%	57.35%	80.43%	34.73%	83.67%	64.04%	98.14%	65.59%	74.39%	79.38%	71.50%	50.10%	74.08%	83.29%	69.74%	95.28%	67.31%	78.11%	80.23%	81.52%	93.61%	17.58%	37.62%	15.47%	55.56%	27.96%	47.04%	90.17%	92.71%	82.18%	77.72%	88.25%	73.31%	66.10%	83.10%	78.71%	79.00%	70.40%	80.15%	53.38%	45.58%	49.84%	49.60%	96.45%	70.89%	81.92%	88.45%	84.43%	98.02%	93.98%	64.46%	72.02%	95.86%	74.47%	83.13%	84.79%	85.07%	75.57%	94.28%	94.71%	86.88%	93.95%	81.54%	99.29%	95.69%	90.85%	92.26%	35.23%	86.48%	85.43%	93.71%	37.98%	67.76%	83.27%	89.93%	94.14%	73.14%	53.94%	83.12%	86.39%	73.10%	84.52%	95.91%	90.37%	70.65%	81.40%	89.92%	76.34%	83.10%	87.95%	90.35%	74.20%	87.19%	87.58%
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100