[fix](search) Make AND/OR/NOT operators case-sensitive in search DSL#59747

airborne12 · 2026-01-12T02:36:12Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #59394

Problem Summary:
The search DSL should only recognize uppercase AND, OR, NOT as boolean operators in search lucene boolean mode. Previously, lowercase and, or, not were also treated as operators, which does not conform to the specification.

This PR makes the boolean operators case-sensitive:

Only uppercase AND, OR, NOT are recognized as operators
Lowercase and, or, not are now treated as regular search terms
Using lowercase operators in DSL will result in a parse error

Release note

Make search DSL boolean operators (AND/OR/NOT) case-sensitive in lucene boolean mode.

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- Yes. Lowercase and, or, not are no longer recognized as operators in search DSL. Users must use uppercase AND, OR, NOT.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2026-01-12T02:36:17Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

airborne12 · 2026-01-12T02:41:28Z

run buildall

airborne12 · 2026-01-12T02:48:28Z

run buildall

doris-robot · 2026-01-12T03:20:04Z

TPC-H: Total hot run time: 31207 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4abe46d4c9305e95d94e6a7775dc86244f3b10a4, data reload: false

------ Round 1 ----------------------------------
q1	17839	4291	4102	4102
q2	2282	362	242	242
q3	10292	1261	701	701
q4	11057	1008	330	330
q5	7610	2005	1902	1902
q6	197	176	143	143
q7	919	791	656	656
q8	9310	1329	1198	1198
q9	4765	4550	4572	4550
q10	6854	1787	1427	1427
q11	1111	297	283	283
q12	1303	747	589	589
q13	18302	3795	3088	3088
q14	287	293	271	271
q15	691	507	495	495
q16	728	671	630	630
q17	687	805	464	464
q18	7738	6481	6237	6237
q19	2230	973	601	601
q20	393	367	247	247
q21	3165	2511	2091	2091
q22	1059	992	960	960
Total cold run time: 108819 ms
Total hot run time: 31207 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4155	4096	4049	4049
q2	326	383	314	314
q3	2072	2540	2179	2179
q4	1326	1750	1321	1321
q5	4061	3990	4294	3990
q6	234	191	145	145
q7	2130	1972	1851	1851
q8	2591	2422	2421	2421
q9	7291	7212	7007	7007
q10	2479	2708	2264	2264
q11	552	468	447	447
q12	723	739	642	642
q13	3561	4060	3250	3250
q14	277	307	270	270
q15	546	503	562	503
q16	724	730	624	624
q17	1153	1292	1382	1292
q18	7955	8086	7901	7901
q19	849	852	854	852
q20	2039	2084	1894	1894
q21	4787	4551	4266	4266
q22	1132	1032	1027	1027
Total cold run time: 50963 ms
Total hot run time: 48509 ms

hello-stephen · 2026-01-12T04:10:22Z

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

airborne12 · 2026-02-13T06:54:05Z

run buildall

airborne12 · 2026-02-13T07:52:39Z

run buildall

doris-robot · 2026-02-13T08:38:20Z

TPC-H: Total hot run time: 28819 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 442d6b154e9412081d48df80b5f700fb7ad107b5, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17694	4420	4318	4318
q2	q3	10646	780	518	518
q4	4675	368	252	252
q5	7535	1218	1035	1035
q6	180	178	145	145
q7	787	818	673	673
q8	9640	1465	1327	1327
q9	5303	4794	4687	4687
q10	6855	1901	1625	1625
q11	492	256	247	247
q12	750	565	456	456
q13	17802	4229	3414	3414
q14	228	232	216	216
q15	960	804	802	802
q16	778	721	673	673
q17	717	854	464	464
q18	5990	5353	5323	5323
q19	1350	975	603	603
q20	508	495	382	382
q21	4568	1892	1413	1413
q22	348	288	246	246
Total cold run time: 97806 ms
Total hot run time: 28819 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4497	4366	4352	4352
q2	q3	1787	2164	1723	1723
q4	839	1144	764	764
q5	4046	4336	4329	4329
q6	181	174	137	137
q7	1706	1564	1486	1486
q8	2429	2654	2540	2540
q9	7588	7377	7396	7377
q10	2669	2879	2479	2479
q11	507	432	445	432
q12	519	627	450	450
q13	3963	4412	3596	3596
q14	295	299	261	261
q15	855	808	808	808
q16	721	755	722	722
q17	1162	1521	1311	1311
q18	7171	6810	6660	6660
q19	900	945	908	908
q20	2141	2271	2060	2060
q21	3979	3579	3348	3348
q22	490	455	406	406
Total cold run time: 48445 ms
Total hot run time: 46149 ms

doris-robot · 2026-02-13T08:49:45Z

TPC-DS: Total hot run time: 184365 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 442d6b154e9412081d48df80b5f700fb7ad107b5, data reload: false

query5	5052	630	501	501
query6	326	221	192	192
query7	4225	466	271	271
query8	337	260	229	229
query9	8712	2715	2702	2702
query10	536	374	328	328
query11	17101	17705	17449	17449
query12	194	126	136	126
query13	1503	477	333	333
query14	7241	3359	3116	3116
query14_1	2929	2824	2866	2824
query15	209	210	191	191
query16	993	488	462	462
query17	1206	739	652	652
query18	2972	456	336	336
query19	206	201	189	189
query20	140	128	127	127
query21	218	135	121	121
query22	5798	5208	4857	4857
query23	17324	16890	16731	16731
query23_1	16819	16866	16765	16765
query24	7131	1639	1248	1248
query24_1	1221	1260	1223	1223
query25	569	469	429	429
query26	1244	255	163	163
query27	2756	478	294	294
query28	4517	1859	1861	1859
query29	827	582	487	487
query30	311	244	213	213
query31	877	737	643	643
query32	82	71	72	71
query33	517	346	288	288
query34	931	922	573	573
query35	645	681	598	598
query36	1119	1171	997	997
query37	141	103	88	88
query38	2975	2873	2918	2873
query39	857	871	809	809
query39_1	796	788	800	788
query40	231	154	136	136
query41	67	65	64	64
query42	107	104	104	104
query43	376	384	362	362
query44	
query45	200	190	185	185
query46	884	986	616	616
query47	2126	2181	2095	2095
query48	333	322	228	228
query49	649	463	398	398
query50	680	276	216	216
query51	4271	4178	4111	4111
query52	108	112	95	95
query53	290	338	284	284
query54	304	274	259	259
query55	89	89	83	83
query56	324	312	326	312
query57	1377	1334	1298	1298
query58	298	277	284	277
query59	2584	2775	2559	2559
query60	358	335	328	328
query61	172	164	200	164
query62	631	594	542	542
query63	306	269	272	269
query64	4808	1253	972	972
query65	
query66	1377	444	353	353
query67	16303	16437	16257	16257
query68	
query69	388	297	281	281
query70	964	950	937	937
query71	332	297	291	291
query72	2804	2868	2425	2425
query73	525	535	318	318
query74	9660	9599	9404	9404
query75	2811	2742	2427	2427
query76	2297	1009	663	663
query77	354	378	298	298
query78	11641	11743	11103	11103
query79	2717	806	590	590
query80	1783	616	517	517
query81	559	292	250	250
query82	999	150	119	119
query83	345	259	239	239
query84	255	128	96	96
query85	884	472	424	424
query86	414	295	297	295
query87	3073	3121	2981	2981
query88	3559	2648	2626	2626
query89	424	368	336	336
query90	1951	171	167	167
query91	159	151	128	128
query92	76	75	70	70
query93	1157	832	502	502
query94	658	319	287	287
query95	589	396	343	343
query96	643	511	226	226
query97	2446	2475	2427	2427
query98	238	213	211	211
query99	1029	994	904	904
Total cold run time: 258298 ms
Total hot run time: 184365 ms

airborne12 · 2026-02-13T11:16:21Z

run buildall

airborne12 · 2026-02-13T11:29:35Z

run buildall

Per specification requirement apache#5, only uppercase AND/OR/NOT should be recognized as operators in search DSL. Lowercase and/or/not should be treated as regular terms, causing parse errors when used as operators. Changes: - Update SearchLexer.g4 to only match uppercase keywords - Update unit tests to expect parse errors for lowercase operators - Update regression tests accordingly - Add comprehensive DSL operator test cases Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

airborne12 · 2026-02-13T12:13:12Z

run buildall

doris-robot · 2026-02-13T12:37:27Z

TPC-H: Total hot run time: 29555 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a318084a2735b2ebd7be8744482f0c971a8f0c1e, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17653	4631	4407	4407
q2	q3	10651	801	513	513
q4	4680	365	263	263
q5	7539	1234	1020	1020
q6	191	194	147	147
q7	847	869	676	676
q8	9285	1607	1471	1471
q9	5066	4950	4935	4935
q10	6803	1899	1677	1677
q11	467	271	231	231
q12	738	583	460	460
q13	17765	4271	3405	3405
q14	237	236	210	210
q15	927	805	787	787
q16	727	724	667	667
q17	738	900	465	465
q18	6085	5415	5364	5364
q19	1109	1124	652	652
q20	519	516	388	388
q21	4689	2083	1533	1533
q22	376	321	284	284
Total cold run time: 97092 ms
Total hot run time: 29555 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4925	4835	4547	4547
q2	q3	1916	2246	1782	1782
q4	999	1284	798	798
q5	4071	4412	4399	4399
q6	214	187	147	147
q7	1903	1670	1518	1518
q8	2837	3060	2854	2854
q9	7579	7650	7569	7569
q10	2761	2880	2461	2461
q11	544	431	414	414
q12	521	613	425	425
q13	4019	4602	3605	3605
q14	277	304	293	293
q15	913	864	806	806
q16	708	765	708	708
q17	1399	1747	1368	1368
q18	7255	6809	6740	6740
q19	1003	965	935	935
q20	2169	2386	2052	2052
q21	4433	3597	3416	3416
q22	510	455	402	402
Total cold run time: 50956 ms
Total hot run time: 47239 ms

doris-robot · 2026-02-13T12:48:48Z

TPC-DS: Total hot run time: 182972 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a318084a2735b2ebd7be8744482f0c971a8f0c1e, data reload: false

query5	4883	643	519	519
query6	335	225	206	206
query7	4223	472	275	275
query8	367	256	236	236
query9	8743	2740	2785	2740
query10	562	388	350	350
query11	17012	17391	16967	16967
query12	217	141	136	136
query13	1296	482	350	350
query14	7014	3368	3060	3060
query14_1	3054	2919	2914	2914
query15	214	223	194	194
query16	1071	515	486	486
query17	1101	731	634	634
query18	2735	442	341	341
query19	213	205	177	177
query20	129	132	129	129
query21	219	138	125	125
query22	4947	4998	4612	4612
query23	17201	16734	16519	16519
query23_1	16645	16679	16691	16679
query24	7144	1634	1239	1239
query24_1	1235	1235	1240	1235
query25	571	475	482	475
query26	1227	256	149	149
query27	2768	479	282	282
query28	4447	1858	1852	1852
query29	799	562	466	466
query30	326	244	210	210
query31	886	721	622	622
query32	82	72	68	68
query33	508	329	280	280
query34	922	899	548	548
query35	641	664	597	597
query36	1060	1145	931	931
query37	125	90	83	83
query38	2931	2915	2866	2866
query39	906	874	829	829
query39_1	824	826	811	811
query40	229	154	137	137
query41	61	57	57	57
query42	103	107	102	102
query43	387	382	354	354
query44	
query45	196	189	180	180
query46	872	986	611	611
query47	2139	2147	2037	2037
query48	319	303	220	220
query49	619	455	375	375
query50	682	272	209	209
query51	4034	4128	4168	4128
query52	105	104	94	94
query53	304	338	287	287
query54	311	276	254	254
query55	83	87	81	81
query56	304	315	310	310
query57	1387	1348	1270	1270
query58	285	279	269	269
query59	2657	2680	2611	2611
query60	349	323	321	321
query61	148	146	144	144
query62	638	590	545	545
query63	313	278	270	270
query64	4840	1243	986	986
query65	
query66	1390	440	346	346
query67	16260	16530	16351	16351
query68	
query69	384	292	282	282
query70	1004	995	932	932
query71	336	318	292	292
query72	2792	2679	2391	2391
query73	535	543	329	329
query74	9980	9869	9768	9768
query75	2823	2709	2441	2441
query76	2318	1034	706	706
query77	367	399	310	310
query78	11192	11574	10712	10712
query79	2847	798	601	601
query80	1803	600	542	542
query81	582	278	245	245
query82	1011	149	120	120
query83	338	257	244	244
query84	256	125	102	102
query85	1029	484	437	437
query86	409	305	292	292
query87	3105	3074	2969	2969
query88	3582	2676	2656	2656
query89	428	377	348	348
query90	1953	170	165	165
query91	164	166	143	143
query92	77	75	75	75
query93	1672	866	498	498
query94	647	306	283	283
query95	581	346	313	313
query96	659	520	229	229
query97	2488	2460	2379	2379
query98	235	220	219	219
query99	1020	997	904	904
Total cold run time: 256772 ms
Total hot run time: 182972 ms

eldenmoon

LGTM

github-actions · 2026-02-14T04:04:06Z

PR approved by at least one committer and no changes requested.

github-actions · 2026-02-14T04:04:08Z

PR approved by anyone and no changes requested.

zzzxl1993

LGTM

…59747) ### What problem does this PR solve? Issue Number: close #xxx Related PR: #59394 Problem Summary: The search DSL should only recognize uppercase `AND`, `OR`, `NOT` as boolean operators in search lucene boolean mode. Previously, lowercase `and`, `or`, `not` were also treated as operators, which does not conform to the specification. This PR makes the boolean operators case-sensitive: - Only uppercase `AND`, `OR`, `NOT` are recognized as operators - Lowercase `and`, `or`, `not` are now treated as regular search terms - Using lowercase operators in DSL will result in a parse error ### Release note Make search DSL boolean operators (AND/OR/NOT) case-sensitive in lucene boolean mode.

…pache#59747) ### What problem does this PR solve? Issue Number: close #xxx Related PR: apache#59394 Problem Summary: The search DSL should only recognize uppercase `AND`, `OR`, `NOT` as boolean operators in search lucene boolean mode. Previously, lowercase `and`, `or`, `not` were also treated as operators, which does not conform to the specification. This PR makes the boolean operators case-sensitive: - Only uppercase `AND`, `OR`, `NOT` are recognized as operators - Lowercase `and`, `or`, `not` are now treated as regular search terms - Using lowercase operators in DSL will result in a parse error ### Release note Make search DSL boolean operators (AND/OR/NOT) case-sensitive in lucene boolean mode.

… search DSL #59747 (#61012) ## Summary - Cherry-pick of #59747 to branch-4.0 - Makes AND/OR/NOT operators case-sensitive in search DSL parsing ## Merge Order This is **PR 1/12** in the search() function pick chain. Should be merged first. ### Check List (For Author) - Test - [x] Regression test - [x] Unit Test - [ ] Manual test - [ ] No need to test - Behavior changed: - [ ] No. - [x] Yes. AND/OR/NOT operators are now case-sensitive (must be uppercase). - Does this need documentation? - [x] No.

…o branch-4.0 Squashed backport of the following master PRs: - apache#59747 [fix](search) Make AND/OR/NOT operators case-sensitive in search DSL - apache#60654 [refactor](search) Refactor SearchDslParser to single-phase ANTLR parsing and fix ES compatibility issues - apache#60782 [fix](search) Upgrade query type for variant subcolumns with analyzer-based indexes - apache#60784 [fix](search) Fix MATCH_ALL_DOCS query failing in multi-field search mode - apache#60786 [feat](search) Support field-grouped query syntax field:(term1 OR term2) - apache#60790 [fix](search) Add searcher cache reuse and DSL result cache for search() function - apache#60793 [fix](search) Fix wildcard query on variant subcolumns returning empty results - apache#60798 [fix](search) Use FE-provided analyzer key for multi-index columns in search() - apache#60814 [fix](search) Fix implicit conjunction incorrectly modifying preceding term in lucene mode - apache#60834 [test](search) Add regression test for wildcard query on variant subcolumns with multi-index - apache#60873 [fix](search) fix MATCH_ALL_DOCS losing occur attribute in multi-field expansion - apache#60891 [fix](search) inject MATCH_ALL_DOCS for multi-MUST_NOT queries in lucene mode

… bug fixes (#61028) ### What problem does this PR solve? Squashed backport of all search() function improvements and bug fixes from master to branch-4.0. This PR combines the following master PRs into a single backport: | Master PR | Type | Description | |-----------|------|-------------| | #59747 | fix | Make AND/OR/NOT operators case-sensitive in search DSL | | #60654 | refactor | Refactor SearchDslParser to single-phase ANTLR parsing and fix ES compatibility issues | | #60782 | fix | Upgrade query type for variant subcolumns with analyzer-based indexes | | #60784 | fix | Fix MATCH_ALL_DOCS query failing in multi-field search mode | | #60786 | feat | Support field-grouped query syntax field:(term1 OR term2) | | #60790 | fix | Add searcher cache reuse and DSL result cache for search() function | | #60793 | fix | Fix wildcard query on variant subcolumns returning empty results | | #60798 | fix | Use FE-provided analyzer key for multi-index columns in search() | | #60814 | fix | Fix implicit conjunction incorrectly modifying preceding term in lucene mode | | #60834 | test | Add regression test for wildcard query on variant subcolumns with multi-index | | #60873 | fix | fix MATCH_ALL_DOCS losing occur attribute in multi-field expansion | | #60891 | fix | inject MATCH_ALL_DOCS for multi-MUST_NOT queries in lucene mode | ### Release note Backport search() function improvements including DSL parser refactoring, multi-field search fixes, variant subcolumn support, query caching, and field-grouped query syntax. ### Check List (For Author) - Test - [x] Regression test - [x] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [x] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason - Behavior changed: - [ ] No. - [x] Yes. New search() function features and bug fixes backported from master. - Does this need documentation? - [x] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label

airborne12 force-pushed the search1 branch from ebc45c7 to 4abe46d Compare January 12, 2026 02:45

airborne12 force-pushed the search1 branch from 4abe46d to 2d0c8ad Compare February 13, 2026 06:53

airborne12 force-pushed the search1 branch from 2d0c8ad to 442d6b1 Compare February 13, 2026 07:50

airborne12 force-pushed the search1 branch from 442d6b1 to f072f62 Compare February 13, 2026 11:14

airborne12 force-pushed the search1 branch from f072f62 to d0f5898 Compare February 13, 2026 11:29

airborne12 force-pushed the search1 branch from d0f5898 to a318084 Compare February 13, 2026 12:13

eldenmoon approved these changes Feb 14, 2026

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 14, 2026

github-actions bot added the reviewed label Feb 14, 2026

zzzxl1993 approved these changes Feb 14, 2026

View reviewed changes

airborne12 merged commit 4128eb4 into apache:master Feb 14, 2026
28 of 30 checks passed

airborne12 deleted the search1 branch February 14, 2026 08:12

airborne12 added the dev/4.0.x label Feb 15, 2026

github-actions bot mentioned this pull request Feb 15, 2026

branch-4.0: [fix](search) Make AND/OR/NOT operators case-sensitive in search DSL #59747 #60773

Closed

yiguolei added dev/4.0.4-merged and removed dev/4.0.x labels Mar 4, 2026

airborne12 mentioned this pull request Mar 4, 2026

branch-4.0: [fix](search) Backport search() function improvements and bug fixes #61028

Merged

16 tasks

yiguolei mentioned this pull request Mar 7, 2026

4.0.4 Release Notes #61121

Open

Conversation

airborne12 commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Jan 12, 2026

Uh oh!

airborne12 commented Jan 12, 2026

Uh oh!

airborne12 commented Jan 12, 2026

Uh oh!

doris-robot commented Jan 12, 2026

Uh oh!

hello-stephen commented Jan 12, 2026

FE UT Coverage Report

Uh oh!

airborne12 commented Feb 13, 2026

Uh oh!

airborne12 commented Feb 13, 2026

Uh oh!

doris-robot commented Feb 13, 2026

Uh oh!

doris-robot commented Feb 13, 2026

Uh oh!

airborne12 commented Feb 13, 2026

Uh oh!

airborne12 commented Feb 13, 2026

Uh oh!

airborne12 commented Feb 13, 2026

Uh oh!

doris-robot commented Feb 13, 2026

Uh oh!

doris-robot commented Feb 13, 2026

Uh oh!

eldenmoon left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 14, 2026

Uh oh!

github-actions bot commented Feb 14, 2026

Uh oh!

zzzxl1993 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

airborne12 commented Jan 12, 2026 •

edited

Loading