jianmu
/
openobserve
mirror of https://github.com/openobserve/openobserve
0
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

feat(editor): PromQL catalog from Prometheus's own term tables 

Item 18 and the two remaining section E fixes. VRL is deliberately out.

THE CATALOG. PromQL completion was seven names in an array literal, so irate,
increase, delta, label_replace, most of the *_over_time family and every
grouping modifier were undiscoverable. utils/query/promqlCompletion.ts now
derives 113 entries from @prometheus-io/codemirror-promql's term tables --
Prometheus's own vocabulary, one-line description per term, versioned with the
upstream package instead of with our memory of what Prometheus shipped last
release. monaco-promql already builds its keyword list from exactly these
tables; the dependency was in the tree but undeclared, and is now explicit.

Registering monaco-promql's own completion provider would have been shorter and
is wrong: it labels EVERY term Keyword, so `rate` would get the keyword glyph --
the icon complaint this workstream started from, reintroduced in a second
language. Functions and aggregations are Function here, modifiers are Keyword,
and the symbolic operators (+, ==, =~) are dropped because they are typed, not
completed.

Insertion is the label and nothing else, which is upstream's reasoning and
holds: some PromQL keywords require parentheses, some forbid them, some are
optional, and the tables carry no signature to tell them apart.

TWO GAPS FOUND WHILE WRITING THE TESTS, both fixed here:

  - The list was only ever filled by getSuggestions, which runs on a query
    update -- so Ctrl+Space on a freshly opened PromQL editor offered nothing
    until the user typed a character. It is now seeded at construction.
  - getSuggestions cleared the list before deciding what to show, and two of its
    branches return without refilling it (an untracked cursor is one), so a
    single suggestion pass could empty it again. The catalog is the floor now.

Metrics move to the field sort lane: they are this language's fields, and behind
113 catalog entries is the same as not being offered.

SECTION E. The double-quote scan moves to utils/query/doubleQuoteWarnings.ts as
a pure function and learns what it is reading: comments and string literals are
masked before matching, so `-- level = "error"` and `body = 'he said "hi"'` stop
being flagged. An UNTERMINATED literal is deliberately left unmasked -- `a = 'x"`
is not a string containing a quote, it is the mismatched pair the scan exists to
report, and masking it would hide the evidence. The component keeps the monaco
half: offsets to positions, positions to markers.

disableSuggestionPopup now calls monaco's hideSuggestWidget instead of
synthesizing an Escape KeyboardEvent -- a guess about monaco's internal key
handling that nothing verified, and one that bubbled out of the editor to
anything else listening for Escape.

VERIFIED IN THE RUNNING APP, not just in tests:
  `level = "error"`               -> 1 marker
  `-- level = "error"`            -> none
  `body = 'he said "hi"'`         -> none
  fresh PromQL composable         -> 113 entries, rate=Function, by=Keyword,
                                     docs present, metric sorts above them all

Full suite: 40414 total, 39983 passed, 23 failed -- 9 load-flaky CodeQueryEditor
and 14 synthetics journey specs, all pre-existing and unrelated. type-check,
eslint and prettier clean.
This commit is contained in:
Prabhat Sharma 2026-08-02 18:43:41 -07:00
parent 8d681be106
commit b20c7a8c8b
5 changed files with 320 additions and 55 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
web/package.json
View File

@ -45,6 +45,7 @@
"@openobserve/browser-rum": "0.4.1",
"@openobserve/node-sql-parser": "^0.1.5",
"@openobserve/rrweb-player": "^0.2.1",
"@prometheus-io/codemirror-promql": "^0.311.3",
"@rudderstack/analytics-js": "3.6.0",
"@tanstack/vue-form": "^1.29.1",
"@tanstack/vue-table": "^8.21.3",

55
web/src/components/CodeQueryEditor.vue
View File

@ -113,6 +113,7 @@ import {
wantsNumericColumn,
rankNumericFieldsFirst,
} from "@/utils/query/editorProviders";
import { findDoubleQuoteIssues } from "@/utils/query/doubleQuoteWarnings";
import { loadPromqlLanguage } from "@/utils/query/promqlLanguageDefinition";
import { useStore } from "vuex";
@ -948,13 +949,11 @@ export default defineComponent({
};
const disableSuggestionPopup = () => {
const escEvent = new KeyboardEvent("keydown", {
keyCode: 27,
code: "Escape",
key: "Escape",
bubbles: true,
});
editorRef.value.dispatchEvent(escEvent);
// monaco's own command, which this file already uses elsewhere. The
// synthetic Escape this replaced was a guess about monaco's internal key
// handling that nothing verified, and it bubbled out of the editor to
// anything else listening for Escape.
editorObj?.trigger("disableSuggestionPopup", "hideSuggestWidget", {});
};
const formatDocument = async () => {
@ -1082,43 +1081,23 @@ export default defineComponent({
const model = editorObj.getModel();
if (!model) return;
// Deciding WHAT is wrong lives in utils/query/doubleQuoteWarnings.ts,
// where it is comment- and string-aware and can be tested without an
// editor. What is left here is the monaco half: offsets to positions,
// positions to markers.
const text = model.getValue();
const markers: any[] = [];
// Two patterns are flagged — both only within value position (after a
// SQL comparison/membership operator). FROM "table" / SELECT "col" are
// intentionally NOT matched.
//
// Pattern A — fully double-quoted: field = "value"
// Pattern B — mismatched quotes: field = "value' or field = 'value"
// Capture group 1: the invalid quoted token
const regex =
/(?:NOT\s+LIKE|NOT\s+IN\s*\(|!=|<>|>=|<=|=|>|<|LIKE|IN\s*\()\s*("[^'"]*'|'[^'"]*"|"[^"]*")/gi;
let match;
while ((match = regex.exec(text)) !== null) {
const quotedStr = match[1]; // the invalid quoted token
const startOffset = match.index + match[0].length - quotedStr.length;
const endOffset = startOffset + quotedStr.length;
const startPos = model.getPositionAt(startOffset);
const endPos = model.getPositionAt(endOffset);
const isMixed =
(quotedStr.startsWith('"') && quotedStr.endsWith("'")) ||
(quotedStr.startsWith("'") && quotedStr.endsWith('"'));
markers.push({
const markers = findDoubleQuoteIssues(text).map((issue) => {
const startPos = model.getPositionAt(issue.startOffset);
const endPos = model.getPositionAt(issue.endOffset);
return {
severity: monaco.MarkerSeverity.Warning,
startLineNumber: startPos.lineNumber,
startColumn: startPos.column,
endLineNumber: endPos.lineNumber,
endColumn: endPos.column,
message: isMixed
? "Mismatched quotes. Use matching single quotes for string values."
: "Double quotes are not valid for string values. Use single quotes instead.",
});
}
message: issue.message,
};
});
monaco.editor.setModelMarkers(model, "dq-validation", markers);
};

38
web/src/composables/usePromqlSuggestions.ts
View File

@ -1,5 +1,7 @@
import searchService from "@/services/search";
import { nextTick, ref } from "vue";
import { PROMQL_CATALOG } from "@/utils/query/promqlCompletion";
import { SORT_LANE } from "@/utils/query/sqlCompletion";
import { useStore } from "vuex";
const usePromqlSuggestions = () => {
@ -19,7 +21,10 @@ const usePromqlSuggestions = () => {
},
});
const store = useStore();
const autoCompletePromqlKeywords: any = ref([]);
// Seeded, not empty: this list is otherwise only filled by getSuggestions,
// which runs on a query update — so Ctrl+Space on a freshly opened PromQL
// editor offered nothing at all until the user typed a character.
const autoCompletePromqlKeywords: any = ref([...PROMQL_CATALOG]);
const metricKeywords: any = ref([]);
const parsePromQlQuery = (query: string) => {
@ -163,7 +168,9 @@ const usePromqlSuggestions = () => {
const metricName = parsedQuery?.metricName || "";
const labels = parsedQuery?.label?.labels || {};
autoCompletePromqlKeywords.value = [];
// NOT cleared here. Two of the branches below return without
// refilling it, and an empty list is never the better answer: the
// catalog is the floor.
const startISOTimestamp: any = autoCompleteData.value.dateTime.startTime;
const endISOTimestamp: any = autoCompleteData.value.dateTime.endTime;
// import search service and call search.get_promql_series
@ -213,7 +220,9 @@ const usePromqlSuggestions = () => {
.finally(() => {
if (labelSuggestions) updatePromqlKeywords(labelSuggestions);
else {
autoCompletePromqlKeywords.value = [];
// Back to the catalog rather than to nothing — the labels are
// unavailable, the language is not.
updatePromqlKeywords([]);
autoCompleteData.value.popup.close("");
}
});
@ -250,20 +259,12 @@ const usePromqlSuggestions = () => {
};
const updatePromqlKeywords = async (data: any[]) => {
autoCompletePromqlKeywords.value = [];
const functions = ["sum", "avg_over_time", "rate", "avg", "max", "topk", "histogram_quantile"];
if (!data.length) {
functions.forEach((fun) => {
autoCompletePromqlKeywords.value.push({
label: fun,
kind: "Function",
insertText: fun,
});
});
autoCompletePromqlKeywords.value.push(...metricKeywords.value);
} else {
autoCompletePromqlKeywords.value.push(...data);
}
// A caller with something contextual to show — label names, label values —
// replaces the list outright. Otherwise: the catalog, plus this org's
// metrics, which sort above it.
autoCompletePromqlKeywords.value = data.length
? [...data]
: [...PROMQL_CATALOG, ...metricKeywords.value];
await nextTick();
autoCompleteData.value.popup.open("");
@ -276,6 +277,9 @@ const usePromqlSuggestions = () => {
label: metric.label + (metric.type ? `(${metric.type})` : ""),
kind: "Variable",
insertText: metric.label,
// The field lane. A metric is what the user came to type; behind 107
// catalog entries is the same as not offering it.
sortText: SORT_LANE.field + metric.label,
});
});
};

146
web/src/utils/query/doubleQuoteWarnings.ts Normal file
View File

@ -0,0 +1,146 @@
// Copyright 2026 OpenObserve Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
/**
* Where a query uses double quotes for a string VALUE.
*
* `WHERE level = "error"` parses as a comparison against a COLUMN named
* `error`, which usually exists nowhere, so the query returns nothing and looks
* like missing data. Worth a warning — but the scan used to run over the raw
* text, so it also warned about `-- level = "error"` in a comment and about the
* legal literal `'he said "hi"'`. A warning on valid SQL teaches people to
* ignore the squiggle, including the times it is right.
*
* Pure on purpose: the editor owns the monaco half (offsets to positions,
* positions to markers), and this owns the decision, which is string in,
* offsets out and can be tested as such.
*/
export interface DoubleQuoteIssue {
/** Index of the opening quote in the ORIGINAL text. */
startOffset: number;
/** Index one past the closing quote. */
endOffset: number;
message: string;
}
const MESSAGE_DOUBLE = "Double quotes are not valid for string values. Use single quotes instead.";
const MESSAGE_MIXED = "Mismatched quotes. Use matching single quotes for string values.";
/**
* Blank out everything that is not executable SQL, preserving every offset.
*
* Line comments, block comments and the INTERIOR of single-quoted strings
* become spaces. String delimiters are deliberately kept: `a = "x'` is a
* mismatched pair that has to stay visible to be reported, and it is only the
* text between quotes that must stop matching.
*/
const maskNonCode = (text: string): string => {
const out = text.split("");
let i = 0;
const blank = (index: number) => {
// Newlines survive so line/column mapping downstream stays correct.
if (out[index] !== "\n") out[index] = " ";
};
while (i < text.length) {
const ch = text[i];
const next = text[i + 1];
if (ch === "-" && next === "-") {
while (i < text.length && text[i] !== "\n") blank(i++);
continue;
}
if (ch === "/" && next === "*") {
blank(i++);
blank(i++);
while (i < text.length && !(text[i] === "*" && text[i + 1] === "/")) blank(i++);
// An unterminated block comment swallows the rest, as SQL does.
if (i < text.length) {
blank(i++);
blank(i++);
}
continue;
}
if (ch === "'") {
// Find the terminator first. An UNTERMINATED literal is left exactly as
// written, because `a = 'x"` is not a string containing a quote — it is
// the mismatched pair this scan exists to report, and masking its
// interior would hide the `"` that makes it reportable.
let end = i + 1;
while (end < text.length) {
if (text[end] === "'") {
// A doubled quote is an escaped quote, not the end of the literal.
if (text[end + 1] === "'") {
end += 2;
continue;
}
break;
}
end++;
}
if (end >= text.length) break;
for (let j = i + 1; j < end; j++) blank(j);
i = end + 1; // both delimiters survive
continue;
}
i++;
}
return out.join("");
};
/**
* Two shapes are reported, both only in value position — after a comparison or
* membership operator. `FROM "table"` and `SELECT "col"` are correct quoting of
* an identifier and are never matched.
*
* fully double-quoted: field = "value"
* mismatched: field = "value' or field = 'value"
*/
const VALUE_QUOTE_REGEX =
/(?:NOT\s+LIKE|NOT\s+IN\s*\(|!=|<>|>=|<=|=|>|<|LIKE|IN\s*\()\s*("[^'"]*'|'[^'"]*"|"[^"]*")/gi;
export const findDoubleQuoteIssues = (text: string): DoubleQuoteIssue[] => {
if (!text) return [];
// Matched against the masked copy; offsets index the original, which the
// masking preserves character for character.
const masked = maskNonCode(text);
const issues: DoubleQuoteIssue[] = [];
const regex = new RegExp(VALUE_QUOTE_REGEX.source, VALUE_QUOTE_REGEX.flags);
let match: RegExpExecArray | null;
while ((match = regex.exec(masked)) !== null) {
const quoted = match[1];
const startOffset = match.index + match[0].length - quoted.length;
const isMixed =
(quoted.startsWith('"') && quoted.endsWith("'")) ||
(quoted.startsWith("'") && quoted.endsWith('"'));
issues.push({
startOffset,
endOffset: startOffset + quoted.length,
message: isMixed ? MESSAGE_MIXED : MESSAGE_DOUBLE,
});
}
return issues;
};

135
web/src/utils/query/promqlCompletion.ts Normal file
View File

@ -0,0 +1,135 @@
// Copyright 2026 OpenObserve Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
/**
* The PromQL completion catalog.
*
* PromQL completion used to be seven names in an array literal — sum,
* avg_over_time, rate, avg, max, topk, histogram_quantile — so `irate`,
* `increase`, `label_replace`, most of the *_over_time family and every
* grouping modifier were simply undiscoverable.
*
* The list is not hand-maintained here either. It is derived from the term
* tables in @prometheus-io/codemirror-promql: Prometheus's own vocabulary,
* carrying a one-line description per term, and versioned with the upstream
* package rather than with our memory of what Prometheus added last release.
* monaco-promql (already a dependency, already used here for the tokenizer)
* builds its keyword list from exactly these tables.
*
* Registering monaco-promql's own completion provider would have been the
* shorter path and is the wrong one: it labels EVERY term `Keyword`, so
* functions get the keyword glyph — the icon complaint this workstream started
* from, reintroduced in a second language.
*/
import {
aggregateOpModifierTerms,
aggregateOpTerms,
atModifierTerms,
binOpModifierTerms,
binOpTerms,
functionIdentifierTerms,
} from "@prometheus-io/codemirror-promql/dist/cjs/complete/promql.terms";
import { SORT_LANE, type CompletionKindName } from "./sqlCompletion";
/** Shaped like SqlCompletionEntry so both languages feed one item builder. */
export interface PromqlCompletionEntry {
name: string;
label: string;
kind: CompletionKindName;
insertText: string;
detail?: string;
documentation?: string;
sortText?: string;
}
/** One upstream term. `info` is the human description; `detail` the group. */
interface PromqlTerm {
label: string;
detail?: string;
info?: string;
type?: string;
}
/**
* Insertion is the label and nothing else.
*
* Upstream's reasoning, which holds: some PromQL keywords require parentheses,
* some forbid them, some are optional, and the term tables carry no signature
* to tell them apart. `rate(` would help; `by(` would be wrong; nothing here
* can distinguish the two. The at-modifiers are LABELLED `start()` and `end()`,
* so they insert their own parens without us inventing any.
*/
const toEntry = (
term: PromqlTerm,
kind: CompletionKindName,
lane: string,
fallbackDetail: string,
): PromqlCompletionEntry => ({
name: term.label,
label: term.label,
kind,
insertText: term.label,
// Upstream fills `detail` for functions and aggregations but leaves the
// modifiers blank; an entry with no detail renders a bare name beside
// entries that explain themselves.
detail: term.detail || fallbackDetail,
...(term.info ? { documentation: term.info } : {}),
sortText: lane + term.label,
});
const byLabel = (a: PromqlCompletionEntry, b: PromqlCompletionEntry) =>
a.label < b.label ? -1 : a.label > b.label ? 1 : 0;
/**
* Aggregation operators and function identifiers — everything callable.
*
* Both groups are Function: `sum` and `rate` are the same kind of thing to
* someone typing, whatever Prometheus's grammar calls them.
*/
export const PROMQL_FUNCTIONS: PromqlCompletionEntry[] = [
...(aggregateOpTerms as PromqlTerm[]).map((t) =>
toEntry(t, "Function", SORT_LANE.function, "aggregation"),
),
...(functionIdentifierTerms as PromqlTerm[]).map((t) =>
toEntry(t, "Function", SORT_LANE.function, "function"),
),
].sort(byLabel);
/**
* Modifiers and word-shaped operators.
*
* The symbolic binary operators (+, -, ==, =~ …) are dropped: they are typed,
* not completed, and a dropdown of punctuation is noise. `and`, `or`, `unless`
* and `atan2` are words, so they stay.
*/
export const PROMQL_KEYWORDS: PromqlCompletionEntry[] = [
...(aggregateOpModifierTerms as PromqlTerm[]),
...(binOpModifierTerms as PromqlTerm[]),
...(atModifierTerms as PromqlTerm[]),
...(binOpTerms as PromqlTerm[]).filter((t) => /^[a-z_][a-z0-9_]*$/i.test(t.label)),
]
.map((t) => toEntry(t, "Keyword", SORT_LANE.clause, "modifier"))
.sort(byLabel);
/**
* What the editor receives. Deduplicated by label, functions before modifiers.
*/
export const PROMQL_CATALOG: PromqlCompletionEntry[] = (() => {
const seen = new Set<string>();
return [...PROMQL_FUNCTIONS, ...PROMQL_KEYWORDS].filter((e) =>
seen.has(e.label) ? false : (seen.add(e.label), true),
);
})();