Skip to content

branch-3.1: [fix](inverted index) Fix skipping data reads for columns when index is hit #57461#57475

Merged
morrySnow merged 1 commit intobranch-3.1from
auto-pick-57461-branch-3.1
Nov 3, 2025
Merged

branch-3.1: [fix](inverted index) Fix skipping data reads for columns when index is hit #57461#57475
morrySnow merged 1 commit intobranch-3.1from
auto-pick-57461-branch-3.1

Conversation

@github-actions
Copy link
Copy Markdown
Contributor

Cherry-picked from #57461

…is hit (#57461)

Different subcolumns from variant share the same parent_unique_id, so we
choose to abandon this optimization.
@github-actions github-actions bot requested a review from morrySnow as a code owner October 29, 2025 13:59
@hello-stephen
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Oct 29, 2025
@hello-stephen
Copy link
Copy Markdown
Contributor

run buildall

@doris-robot
Copy link
Copy Markdown

TPC-H: Total hot run time: 33183 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 28f753b9da44c3a9a424dd3f5cb5b46933e96cf9, data reload: false

------ Round 1 ----------------------------------
q1	17625	5569	5393	5393
q2	2026	409	292	292
q3	12079	1256	763	763
q4	10301	883	480	480
q5	8957	2445	2160	2160
q6	188	167	132	132
q7	899	766	625	625
q8	9341	1459	1254	1254
q9	5263	4955	4930	4930
q10	6792	2306	1832	1832
q11	474	292	274	274
q12	349	367	227	227
q13	17799	3653	3066	3066
q14	251	236	219	219
q15	533	470	469	469
q16	447	450	385	385
q17	659	871	396	396
q18	7157	6397	6628	6397
q19	1215	976	547	547
q20	340	354	229	229
q21	3354	2268	2118	2118
q22	1064	1039	995	995
Total cold run time: 107113 ms
Total hot run time: 33183 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5541	5506	5533	5506
q2	244	334	238	238
q3	2324	2669	2312	2312
q4	1422	1826	1384	1384
q5	4445	5154	5077	5077
q6	176	170	133	133
q7	2146	2013	1857	1857
q8	2696	2851	2742	2742
q9	7393	7386	7301	7301
q10	2996	3293	2800	2800
q11	602	534	518	518
q12	689	809	642	642
q13	3475	3781	3196	3196
q14	285	315	283	283
q15	523	473	471	471
q16	461	507	456	456
q17	1288	1770	1307	1307
q18	7670	7529	7530	7529
q19	872	1049	1162	1049
q20	2080	2096	1957	1957
q21	5400	5068	4743	4743
q22	1134	1128	1076	1076
Total cold run time: 53862 ms
Total hot run time: 52577 ms

@doris-robot
Copy link
Copy Markdown

TPC-DS: Total hot run time: 196746 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 28f753b9da44c3a9a424dd3f5cb5b46933e96cf9, data reload: false

query1	980	413	405	405
query2	6240	2081	2066	2066
query3	8686	222	208	208
query4	33499	24142	23917	23917
query5	3785	604	476	476
query6	297	208	182	182
query7	4221	518	337	337
query8	313	258	246	246
query9	9355	2910	2877	2877
query10	469	335	281	281
query11	18325	15521	15322	15322
query12	157	109	116	109
query13	1559	573	463	463
query14	9078	6761	7369	6761
query15	227	192	185	185
query16	8049	644	498	498
query17	1600	812	621	621
query18	2157	435	345	345
query19	241	234	177	177
query20	145	124	121	121
query21	211	135	115	115
query22	4718	4838	4544	4544
query23	35662	34243	34229	34229
query24	8056	2795	2827	2795
query25	558	521	454	454
query26	1205	296	192	192
query27	2045	528	407	407
query28	5416	2491	2435	2435
query29	815	656	492	492
query30	244	200	165	165
query31	1015	931	838	838
query32	90	65	66	65
query33	511	411	364	364
query34	808	911	552	552
query35	824	850	740	740
query36	1092	1150	1047	1047
query37	112	101	73	73
query38	4098	4057	4038	4038
query39	1558	1510	1500	1500
query40	221	127	108	108
query41	52	53	50	50
query42	133	119	118	118
query43	581	600	542	542
query44	1463	934	932	932
query45	191	183	183	183
query46	953	1095	718	718
query47	2016	2008	1941	1941
query48	447	470	400	400
query49	794	522	441	441
query50	723	729	473	473
query51	7554	7415	7512	7415
query52	115	107	99	99
query53	254	269	206	206
query54	602	595	514	514
query55	89	86	85	85
query56	306	297	291	291
query57	1287	1253	1219	1219
query58	257	236	238	236
query59	3230	3352	3228	3228
query60	317	311	297	297
query61	139	120	123	120
query62	825	774	696	696
query63	247	206	212	206
query64	4533	1022	706	706
query65	3383	3333	3385	3333
query66	1122	411	330	330
query67	16648	16203	15779	15779
query68	7873	891	601	601
query69	502	328	283	283
query70	1276	1157	1248	1157
query71	394	326	275	275
query72	5731	3770	3785	3770
query73	703	783	387	387
query74	10331	9391	9227	9227
query75	3287	3205	2726	2726
query76	3107	1202	786	786
query77	703	378	287	287
query78	10512	10537	9808	9808
query79	3087	889	644	644
query80	744	539	464	464
query81	524	294	224	224
query82	554	125	94	94
query83	183	169	154	154
query84	288	100	79	79
query85	793	375	319	319
query86	355	310	316	310
query87	4412	4437	4333	4333
query88	4402	2673	2605	2605
query89	416	345	308	308
query90	1849	195	196	195
query91	146	140	116	116
query92	71	57	55	55
query93	1015	927	605	605
query94	684	417	324	324
query95	355	290	281	281
query96	539	621	310	310
query97	3180	3286	3204	3204
query98	234	217	203	203
query99	1592	1437	1304	1304
Total cold run time: 296745 ms
Total hot run time: 196746 ms

@doris-robot
Copy link
Copy Markdown

ClickBench: Total hot run time: 28.9 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 28f753b9da44c3a9a424dd3f5cb5b46933e96cf9, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.61	0.10	0.11
query5	0.50	0.52	0.53
query6	1.13	0.73	0.73
query7	0.03	0.02	0.02
query8	0.05	0.03	0.04
query9	0.58	0.51	0.52
query10	0.56	0.58	0.57
query11	0.15	0.10	0.10
query12	0.14	0.11	0.11
query13	0.63	0.59	0.60
query14	0.80	0.81	0.80
query15	0.90	0.84	0.84
query16	0.41	0.40	0.41
query17	1.05	1.02	1.05
query18	0.24	0.23	0.23
query19	1.98	1.84	1.95
query20	0.02	0.01	0.01
query21	15.37	1.02	0.68
query22	0.75	0.79	0.60
query23	15.13	1.47	0.64
query24	3.73	0.44	0.60
query25	0.15	0.17	0.05
query26	0.39	0.18	0.14
query27	0.05	0.05	0.06
query28	12.81	1.13	0.45
query29	12.64	4.09	3.36
query30	0.28	0.11	0.07
query31	2.82	0.62	0.38
query32	3.24	0.55	0.46
query33	3.05	3.08	3.05
query34	16.63	5.24	4.56
query35	4.63	4.60	4.58
query36	0.67	0.51	0.49
query37	0.09	0.06	0.07
query38	0.05	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.14	0.12
query41	0.08	0.04	0.03
query42	0.04	0.03	0.03
query43	0.03	0.03	0.03
Total cold run time: 103.98 s
Total hot run time: 28.9 s

@morrySnow morrySnow merged commit 98f4301 into branch-3.1 Nov 3, 2025
21 of 22 checks passed
@github-actions github-actions bot deleted the auto-pick-57461-branch-3.1 branch November 3, 2025 03:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants