LongShOT Leaderboard

106 models

Rankings

Model Leaderboard

Performance rankings across 4 task categories on LongShOTBench. All scores in %.

Last updated: June 4, 2026

CP Core PerceptionRE ReasoningInfo InformationMM Multimodal
Score Category Modality Duration
#ModelParamsInputOverallCPREInfoMMVisualAudioSpeechShortMedLong
LongShOTAgent
LongShOTAgent
Agentic Systems
AgentV+A66.64
55.2875.6969.5766.0064.9657.6166.7666.4365.6063.50
Qwen3-Omni-30B-A3B-T
Qwen3-Omni-30B-A3B-T
Omni-modal Models
30B (3B active)V+A64.05
51.4472.8665.4066.5062.0955.8963.7462.2262.9360.07
Gemini 3.1 Pro Preview
Gemini 3.1 Pro Preview
Proprietary (Closed-source API)
APIV+A55.63
43.0165.8255.5758.1552.0546.1553.6550.0551.9652.57
4Voxtral-Mini-3B-2507
Voxtral-Mini-3B-2507
Audio-Only LLMs
3BAudio49.27
33.9253.3142.9040.4141.4335.0842.8537.5841.2242.40
5Voxtral-Small-24B-2507
Voxtral-Small-24B-2507
Audio-Only LLMs
24BAudio48.41
32.6351.6541.3843.5340.3933.1041.6335.8740.1941.13
6Qwen3-VL-235B-A22B-T
Qwen3-VL-235B-A22B-T
Video-native VLMs
235B (22B active)Video47.23
39.8453.1245.3650.6045.4843.9946.2243.6844.3148.04
7Qwen3-VL-32B-T
Qwen3-VL-32B-T
Video-native VLMs
32BVideo46.16
38.5750.7944.3250.9744.1343.1044.8143.0543.5345.49
8Vero-Qwen3I-8B
Vero-Qwen3I-8B
Video-native VLMs
8BVideo45.73
40.5449.3442.2950.7643.5743.1243.9944.5843.3244.09
9Kimi-K2.6
Kimi-K2.6
Video-native VLMs
1.1TVideo45.20
37.5251.6142.7548.9142.8841.4943.6243.3243.1542.12
10Qwen3-VL-30B-A3B-T
Qwen3-VL-30B-A3B-T
Video-native VLMs
30B (3B active)Video43.40
35.8949.0141.4447.2641.2939.7842.0640.3740.5543.00
11Qwen3-VL-235B-A22B-I
Qwen3-VL-235B-A22B-I
Video-native VLMs
235B (22B active)Video42.19
35.6745.6140.1147.3740.0538.8040.5445.4239.4441.20
12Qwen3-VL-8B-T
Qwen3-VL-8B-T
Video-native VLMs
8BVideo41.98
34.6147.2238.9847.1039.4838.2740.1241.7338.9240.69
13Gemma-3-27B
Gemma-3-27B
Frame-based VLMs
27BFrames41.55
34.6747.2039.1445.2239.3437.4139.9940.6338.2641.71
14Intern-S1-mini
Intern-S1-mini
Video-native VLMs
14BVideo41.26
34.0045.9839.4145.6739.1437.5539.7038.7838.4840.72
15Nemotron-3-Nano-Omni-30B-A3B-R
Nemotron-3-Nano-Omni-30B-A3B-R
Omni-modal Models
30B (3B active)V+A40.96
30.9050.3942.0040.5538.8932.8939.9843.5238.9138.62
16Qwen3-VL-32B-I
Qwen3-VL-32B-I
Video-native VLMs
32BVideo40.26
34.5543.7236.9245.8437.7836.3538.1642.0437.7837.43
17Gemma-3-12B
Gemma-3-12B
Frame-based VLMs
12BFrames39.58
32.0846.0636.6943.4636.9235.4437.6034.4236.0139.11
18Qwen3-Omni-30B-A3B-I
Qwen3-Omni-30B-A3B-I
Omni-modal Models
30B (3B active)V+A39.17
31.2146.9236.9641.5936.2732.7737.2029.6436.1836.89
19Step3-VL-10B
Step3-VL-10B
Video-native VLMs
10BVideo38.65
31.3544.4034.7844.0936.1034.3536.6036.2035.2138.04
20Gemma-4-31B
Gemma-4-31B
Frame-based VLMs
31BFrames38.49
31.2343.5936.0143.1435.9934.6736.5140.8335.3736.85
21Qwen3-VL-4B-T
Qwen3-VL-4B-T
Video-native VLMs
4BVideo38.11
30.7742.9435.4443.2835.6134.3936.0737.7635.1436.51
22GLM-4.5V
GLM-4.5V
Video-native VLMs
106B (12B active)Video37.66
31.1042.5034.5342.5335.4132.6636.0137.3434.5437.27
23MiMo-VL-7B-RL-2508
MiMo-VL-7B-RL-2508
Video-native VLMs
7BVideo37.56
29.9242.5434.7743.0334.8633.1935.4535.1334.3635.90
24Qwen2.5-VL-32B-I
Qwen2.5-VL-32B-I
Video-native VLMs
32BVideo35.90
30.8739.2433.1940.3133.9631.7034.3935.7733.3535.25
25Gemma-3-4B
Gemma-3-4B
Frame-based VLMs
4BFrames35.61
28.8940.4232.8740.2733.1931.2733.7235.6032.4534.77
26Qwen3-VL-8B-I
Qwen3-VL-8B-I
Video-native VLMs
8BVideo34.47
28.7937.5031.0040.5731.3630.2731.7735.6331.1531.69
27Gemma-4-26B-A4B
Gemma-4-26B-A4B
Frame-based VLMs
25B (3.8B active)Frames34.28
29.4439.3731.7036.6132.3431.3832.6934.9332.6131.29
28Ovis2.6-30B-A3B
Ovis2.6-30B-A3B
Video-native VLMs
30B (3B active)Video33.54
26.6136.6031.1139.8831.1529.2431.6336.1130.7731.64
29GLM-4.6V-Flash
GLM-4.6V-Flash
Video-native VLMs
9BVideo33.35
26.7237.1130.6438.9330.9129.4731.4232.5530.0832.75
30Qwen3.6-27B
Qwen3.6-27B
Video-native VLMs
27BVideo32.87
24.5937.6130.3638.9229.6928.3030.2628.0429.4530.24
31Qwen2.5-VL-72B-I
Qwen2.5-VL-72B-I
Video-native VLMs
72BVideo32.33
26.4535.2929.2738.3130.0127.4830.4229.7729.3631.44
32Qwen3.6-35B-A3B
Qwen3.6-35B-A3B
Video-native VLMs
35B (3B active)Video31.74
22.6036.8727.7339.7527.6325.9328.2232.0527.3927.98
33Qwen3.5-27B
Qwen3.5-27B
Video-native VLMs
27BVideo31.41
22.3937.8528.4136.9928.2125.5028.8229.3427.6529.49
34Qwen3.5-122B-A10B
Qwen3.5-122B-A10B
Video-native VLMs
122B (10B active)Video31.12
22.5034.9829.3637.6327.8225.4428.4233.2727.3728.56
35Qwen3.5-35B-A3B
Qwen3.5-35B-A3B
Video-native VLMs
35B (3B active)Video31.08
22.1736.2327.1138.8226.9824.8327.5929.3026.4528.14
36R-4B
R-4B
Video-native VLMs
4BVideo31.02
24.3235.7728.0435.9528.4927.0228.9134.8527.3630.82
37Fun-Audio-Chat-8B
Fun-Audio-Chat-8B
Audio-Only LLMs
8BAudio30.94
19.9228.8522.9330.6924.0022.1324.5624.9023.8324.40
38Qwen3-VL-30B-A3B-I
Qwen3-VL-30B-A3B-I
Video-native VLMs
30B (3B active)Video30.68
26.8032.8227.7835.3229.0128.0829.3632.9428.7229.45
39GLM-4.1V-9B-T
GLM-4.1V-9B-T
Video-native VLMs
9BVideo29.80
22.9235.0226.8334.4427.3325.8227.7327.2226.7428.72
40InternVL3.5-241B-A28B
InternVL3.5-241B-A28B
Video-native VLMs
241B (28B active)Video29.77
24.6332.7426.6335.0727.3125.4127.7130.6626.8128.30
41InternVL3-78B
InternVL3-78B
Video-native VLMs
78BVideo29.40
23.8531.7826.9735.0126.7624.8427.1130.7226.3527.38
42InternVL3-38B
InternVL3-38B
Video-native VLMs
38BVideo29.36
24.9332.9825.8133.7226.7824.8027.0930.1726.3727.59
43Qwen3-VL-4B-I
Qwen3-VL-4B-I
Video-native VLMs
4BVideo28.68
23.8430.8525.3834.6526.0624.8826.3431.8625.7126.72
44VideoRFT-7B
VideoRFT-7B
Video-native VLMs
7BVideo28.56
24.3729.5725.9234.3826.8925.3227.1128.8526.6527.27
45DeepEyes-7B
DeepEyes-7B
Video-native VLMs
7BVideo28.07
23.4030.8224.1833.9026.0324.5026.3229.5225.4727.17
46Ovis2.5-9B
Ovis2.5-9B
Video-native VLMs
9BVideo28.03
22.3929.9824.9034.8625.1623.9925.4829.2424.5826.40
47Video-R2
Video-R2
Video-native VLMs
7BVideo27.83
23.0129.2825.2233.7925.9124.7826.1924.4025.6426.63
48Molmo2-8B
Molmo2-8B
Video-native VLMs
8BVideo27.81
22.6229.4824.1135.0125.3224.2125.5031.9325.1025.45
49Video-R1-7B
Video-R1-7B
Video-native VLMs
7BVideo27.72
22.4429.0424.6734.7325.4324.4225.6825.8824.9726.51
50LongVT-RL
LongVT-RL
Video-native VLMs
7BVideo27.68
22.6128.8025.2834.0525.5925.4725.7727.3825.4625.69
51RePlan-Qwen2.5-VL-7B
RePlan-Qwen2.5-VL-7B
Video-native VLMs
7BVideo27.63
22.7830.2123.4034.1625.3423.9125.6127.4925.0525.91
52InternVL3-14B
InternVL3-14B
Video-native VLMs
14BVideo27.54
23.2630.2124.1032.6225.4123.4825.7125.9824.8826.58
53Qwen3.5-9B
Qwen3.5-9B
Video-native VLMs
9BVideo27.47
18.9733.1824.6633.1023.5721.8124.0425.8823.3124.05
54TimeSearch-R
TimeSearch-R
Video-native VLMs
7BVideo27.18
22.2230.4323.9332.1625.2823.7025.5324.3724.9726.05
55ERNIE-4.5-VL-28B-A3B
ERNIE-4.5-VL-28B-A3B
Video-native VLMs
28B (3B active)Video27.13
23.1030.6424.0630.7225.7224.5826.0826.6525.0227.29
56Gemma-4-E4B
Gemma-4-E4B
Omni-modal Models
8B (4.5B eff)V+A26.90
22.2727.5624.4133.3425.1824.7725.4830.3624.9025.47
57InternVL3.5-8B
InternVL3.5-8B
Video-native VLMs
8BVideo26.58
20.9427.9923.3034.0923.6022.1223.8425.4723.3224.08
58VideoChat-R1-7B
VideoChat-R1-7B
Video-native VLMs
7BVideo26.40
21.5927.5623.5232.9424.7523.3024.9922.1324.2625.96
59InternVL3.5-38B
InternVL3.5-38B
Video-native VLMs
38BVideo26.23
21.4628.5022.6332.3423.9222.5124.2127.5323.5224.67
60PixelReasoner-RL-v1
PixelReasoner-RL-v1
Video-native VLMs
7BVideo26.14
21.2229.5622.8430.9424.0222.6024.2725.5823.6424.89
61Qwen2.5-VL-7B-I
Qwen2.5-VL-7B-I
Video-native VLMs
7BVideo25.96
21.7327.2923.6731.1624.2423.0924.5326.7623.9324.85
62InternVL2.5-78B
InternVL2.5-78B
Video-native VLMs
78BVideo25.71
21.2227.5322.0832.0422.9921.5623.2925.7022.4924.05
63Ovis2-16B
Ovis2-16B
Frame-based VLMs
16BFrames25.32
20.2828.1620.8731.9822.8621.8423.1725.7222.5123.57
64InternVL3-8B
InternVL3-8B
Video-native VLMs
8BVideo24.68
20.5225.7021.6630.8322.3621.3822.5225.3321.9923.09
65InternVL2.5-38B
InternVL2.5-38B
Video-native VLMs
38BVideo24.49
20.7026.4619.9830.7922.0720.8522.4422.6221.4923.44
66Kimi-VL-A3B-T
Kimi-VL-A3B-T
Frame-based VLMs
8B (3B active)Frames24.32
17.7627.4921.2430.7721.2319.5721.6723.3221.0421.43
67R1-Onevision-7B
R1-Onevision-7B
Video-native VLMs
7BVideo23.89
18.7826.2620.5829.9321.9920.7322.2422.0121.7122.62
68Phi-4-R-Vision-15B
Phi-4-R-Vision-15B
Video-native VLMs
15BVideo23.61
19.3325.9019.2130.0121.1820.5921.4325.4520.3922.74
69Qwen2.5-VL-7B-VRAG
Qwen2.5-VL-7B-VRAG
Video-native VLMs
7BVideo23.46
20.4523.2820.7429.3822.3222.2822.4919.8621.9523.26
70AURA-32B
AURA-32B
Video-native VLMs
32BVideo23.19
19.5724.2520.2428.7021.0920.5921.2521.3820.8121.73
71InternVL3.5-4B-I
InternVL3.5-4B-I
Video-native VLMs
4BVideo22.99
19.0624.1519.7329.0221.0319.5121.2720.8820.5822.12
72Audio Flamingo 3
Audio Flamingo 3
Audio-Only LLMs
8BAudio22.90
13.5018.1814.5422.6715.3212.3515.7619.3515.3715.17
73MiniCPM-o 4.5
MiniCPM-o 4.5
Omni-modal Models
8BV+A22.82
20.4525.9023.1721.7422.3821.4322.6722.0322.0723.12
74Gemma-4-E2B
Gemma-4-E2B
Omni-modal Models
5B (2.3B eff)V+A22.44
19.2026.0320.8823.6722.0220.8922.2731.6621.8921.72
75Qwen3.5-4B
Qwen3.5-4B
Video-native VLMs
4BVideo22.40
16.2626.5620.3126.4819.9918.6620.3022.5220.0719.48
76EchoInk-R1-7B
EchoInk-R1-7B
Omni-modal Models
7BV+A22.34
18.1023.0219.9528.2820.2818.2920.6522.7020.4020.06
77Keye-VL-1.5-8B
Keye-VL-1.5-8B
Video-native VLMs
8BVideo22.33
17.1924.8318.9728.3419.9019.4820.1722.5619.3521.00
78HumanSense Omni-R
HumanSense Omni-R
Omni-modal Models
7BV+A22.23
18.1723.3019.8227.6120.3417.9620.7122.8120.1120.99
79NVIDIA Nemotron Nano 12B VL
NVIDIA Nemotron Nano 12B VL
Video-native VLMs
12BVideo22.04
17.3822.9518.7829.0519.4618.8319.7916.5818.9320.79
80Qwen2.5-Omni-7B
Qwen2.5-Omni-7B
Omni-modal Models
7BV+A21.97
17.6922.7220.2727.2020.1218.1720.4821.8020.2220.00
81MiniCPM-V 4.5
MiniCPM-V 4.5
Video-native VLMs
8BVideo21.66
17.5923.6318.6126.8019.3719.0019.7021.7918.9220.43
82SpaceThinker-3B
SpaceThinker-3B
Video-native VLMs
3BVideo20.21
16.9421.6918.7123.5218.8917.4719.0719.9818.2520.33
83Qwen3-VL-2B-I
Qwen3-VL-2B-I
Video-native VLMs
2BVideo19.98
17.3520.0717.3025.2218.5318.0018.6521.2418.0719.53
84Ovis2-8B
Ovis2-8B
Frame-based VLMs
8BFrames19.93
15.8121.5117.2625.1417.6517.5517.8418.1516.9919.15
85Qwen2.5-VL-3B-I
Qwen2.5-VL-3B-I
Video-native VLMs
3BVideo19.78
16.2621.1016.6125.1517.9717.3618.1514.6317.6119.04
86Qwen3.5-2B
Qwen3.5-2B
Video-native VLMs
2BVideo19.74
17.2519.1218.5224.0718.1417.5518.3118.2317.9118.61
87Baichuan-Omni 1.5
Baichuan-Omni 1.5
Omni-modal Models
7BV+A19.47
15.9819.4615.7526.7317.3116.6017.4718.3516.8018.44
88Qwen2.5-Omni-3B
Qwen2.5-Omni-3B
Omni-modal Models
3BV+A19.26
15.8619.3916.7925.0117.3515.7417.6619.3117.0718.05
89MiniCPM-o 2.6
MiniCPM-o 2.6
Omni-modal Models
8BV+A19.19
15.3819.2316.2225.9417.0016.2717.2214.3116.8017.62
90SALMONN2+ 7B
SALMONN2+ 7B
Omni-modal Models
7BV+A17.99
14.0019.6715.9122.3715.9113.4016.3016.1715.7316.49
91Ovis2-4B
Ovis2-4B
Frame-based VLMs
4BFrames17.76
14.4118.1114.9723.5415.5615.5215.6516.0114.8817.11
92Qwen3.5-0.8B
Qwen3.5-0.8B
Video-native VLMs
0.8BVideo17.44
15.1016.6015.6722.3916.0515.7316.1815.6915.8416.57
93VideoLLaMA3-7B
VideoLLaMA3-7B
Video-native VLMs
7BVideo16.47
12.4716.7213.6223.0513.7413.3213.8813.2013.3114.76
94OmniVinci-9B
OmniVinci-9B
Omni-modal Models
9BV+A16.25
13.3416.7514.5920.3314.0913.5614.3815.5314.0314.17
95Kimi-VL-A3B-I
Kimi-VL-A3B-I
Frame-based VLMs
8B (3B active)Frames15.76
12.3816.5812.2921.7913.5313.2613.7611.2713.2314.26
96OmniAtlas-Qwen2.5-7B
OmniAtlas-Qwen2.5-7B
Omni-modal Models
7BV+A15.43
14.1215.9515.8615.7915.5115.0315.7516.8315.5515.43
97Monet-7B
Monet-7B
Video-native VLMs
7BVideo13.89
10.7414.8311.2818.7012.2712.1512.4310.8411.7613.54
98Qwen2-Audio-7B-I
Qwen2-Audio-7B-I
Audio-Only LLMs
7BAudio13.18
6.047.616.1612.477.006.887.146.196.647.92
99LLaVA-OneVision-7B
LLaVA-OneVision-7B
Video-native VLMs
7BVideo13.07
11.1213.4311.4216.3111.7911.3211.9813.0411.6212.20
100VideoMind
VideoMind
Agentic Systems
AgentV+A10.83
8.229.908.7416.438.777.038.7613.378.828.56
101Tempo-6B
Tempo-6B
Video-native VLMs
6BVideo9.97
6.329.667.1616.747.285.457.3311.077.237.38
102SenseNova-SI-1.3-8B
SenseNova-SI-1.3-8B
Video-native VLMs
8BVideo9.78
7.6210.037.4813.978.508.818.696.008.019.73
103OneThinker-8B
OneThinker-8B
Video-native VLMs
8BVideo7.09
6.677.728.555.397.367.507.486.176.788.69
104Video-RAG
Video-RAG
Agentic Systems
AgentV+A6.66
5.415.997.357.886.124.506.349.486.076.31
105Vgent
Vgent
Agentic Systems
AgentV+A5.83
3.445.434.2910.154.133.074.246.504.233.98
106VideoScore2
VideoScore2
Video-native VLMs
7BVideo2.77
2.643.272.592.602.762.542.684.502.862.46