Skip to content

Commit 09c713c

Browse files
committed
Docs: update Java customization doc to cover JSON extensions
1 parent 2bb1fd0 commit 09c713c

1 file changed

Lines changed: 143 additions & 72 deletions

File tree

‎docs/codeql/codeql-language-guides/customizing-library-models-for-java-and-kotlin.rst‎

Lines changed: 143 additions & 72 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,29 @@ Syntax used to define an element in an extension file
3030
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3131

3232
Each model of an element is defined using a data extension where each tuple constitutes a model.
33-
A data extension file to extend the standard Java queries included with CodeQL is a YAML file with the form:
33+
A data extension file to extend the standard Java queries included with CodeQL can be written using either JSON or YAML. The JSON format takes the following form:
34+
35+
.. code-block:: json
36+
37+
{
38+
"extensions": [
39+
{
40+
"addsTo": {
41+
"pack": "codeql/java-all",
42+
"extensible": "<name of extensible predicate>"
43+
},
44+
"data": [
45+
["tuple", 1],
46+
["tuple", 2]
47+
// ...
48+
]
49+
}
50+
]
51+
}
52+
53+
Files in the JSON format must use the ``.json`` file extenion. Single-line (``//``) and multi-line (``/* ... */``) comments are supported as a non-standard JSON extension.
54+
55+
A YAML file has the following form:
3456

3557
.. code-block:: yaml
3658
@@ -39,11 +61,11 @@ A data extension file to extend the standard Java queries included with CodeQL i
3961
pack: codeql/java-all
4062
extensible: <name of extensible predicate>
4163
data:
42-
- <tuple1>
43-
- <tuple2>
64+
- ["tuple", 1]
65+
- ["tuple", 2]
4466
- ...
4567
46-
Each YAML file may contain one or more top-level extensions.
68+
Each data extension file may contain one or more top-level extensions.
4769

4870
- ``addsTo`` defines the CodeQL pack name and extensible predicate that the extension is injected into.
4971
- ``data`` defines one or more rows of tuples that are injected as values into the extensible predicate. The number of columns and their types must match the definition of the extensible predicate.
@@ -74,9 +96,9 @@ Specifying types in Java and Kotlin models
7496

7597
**Nested and inner classes** are denoted by joining the enclosing type and the nested type with a dollar sign (``$``), for example ``Outer$Inner``. This applies both to the type column and to nested types in a signature. For example, the ``Level`` enum nested inside the ``Logger`` interface, nested inside the ``System`` class, is written as ``System$Logger$Level``:
7698

77-
.. code-block:: yaml
99+
.. code-block:: json
78100
79-
- ["java.lang", "System$Logger", True, "log", "(System$Logger$Level,String)", "", "Argument[1]", "log-injection", "manual"]
101+
["java.lang", "System$Logger", true, "log", "(System$Logger$Level,String)", "", "Argument[1]", "log-injection", "manual"]
80102
81103
**Generics** are erased, so type parameters are removed:
82104

@@ -85,9 +107,9 @@ Specifying types in Java and Kotlin models
85107

86108
For example, ``forEach`` on ``Iterable<T>`` takes a ``Consumer<? super T>`` argument, so the type is ``Iterable`` and the signature is ``(Consumer)``:
87109

88-
.. code-block:: yaml
110+
.. code-block:: json
89111
90-
- ["java.lang", "Iterable", True, "forEach", "(Consumer)", "", "Argument[this].Element", "Argument[0].Parameter[0]", "value", "manual"]
112+
["java.lang", "Iterable", true, "forEach", "(Consumer)", "", "Argument[this].Element", "Argument[0].Parameter[0]", "value", "manual"]
91113
92114
Examples of custom model definitions
93115
------------------------------------
@@ -109,20 +131,27 @@ This is the ``execute`` method in the ``Statement`` class, which is located in t
109131
110132
We need to add a tuple to the ``sinkModel(package, type, subtypes, name, signature, ext, input, kind, provenance)`` extensible predicate by updating a data extension file.
111133

112-
.. code-block:: yaml
113-
114-
extensions:
115-
- addsTo:
116-
pack: codeql/java-all
117-
extensible: sinkModel
118-
data:
119-
- ["java.sql", "Statement", True, "execute", "(String)", "", "Argument[0]", "sql-injection", "manual"]
134+
.. code-block:: json
135+
136+
{
137+
"extensions": [
138+
{
139+
"addsTo": {
140+
"pack": "codeql/java-all",
141+
"extensible": "sinkModel"
142+
},
143+
"data": [
144+
["java.sql", "Statement", true, "execute", "(String)", "", "Argument[0]", "sql-injection", "manual"]
145+
]
146+
}
147+
]
148+
}
120149
121150
The first five values identify the callable (in this case a method) to be modeled as a sink.
122151

123152
- The first value ``java.sql`` is the package name.
124153
- The second value ``Statement`` is the name of the class (type) that contains the method.
125-
- The third value ``True`` is a flag that indicates whether or not the model also applies to all overrides of the method.
154+
- The third value ``true`` is a flag that indicates whether or not the model also applies to all overrides of the method.
126155
- The fourth value ``execute`` is the method name.
127156
- The fifth value ``(String)`` is the method input type signature.
128157

@@ -147,20 +176,27 @@ This is the ``getInputStream`` method in the ``Socket`` class, which is located
147176
148177
We need to add a tuple to the ``sourceModel(package, type, subtypes, name, signature, ext, output, kind, provenance)`` extensible predicate by updating a data extension file.
149178

150-
.. code-block:: yaml
151-
152-
extensions:
153-
- addsTo:
154-
pack: codeql/java-all
155-
extensible: sourceModel
156-
data:
157-
- ["java.net", "Socket", False, "getInputStream", "()", "", "ReturnValue", "remote", "manual"]
179+
.. code-block:: json
180+
181+
{
182+
"extensions": [
183+
{
184+
"addsTo": {
185+
"pack": "codeql/java-all",
186+
"extensible": "sourceModel"
187+
},
188+
"data": [
189+
["java.net", "Socket", false, "getInputStream", "()", "", "ReturnValue", "remote", "manual"]
190+
]
191+
}
192+
]
193+
}
158194
159195
The first five values identify the callable (in this case a method) to be modeled as a source.
160196

161197
- The first value ``java.net`` is the package name.
162198
- The second value ``Socket`` is the name of the class (type) that contains the source.
163-
- The third value ``False`` is a flag that indicates whether or not the model also applies to all overrides of the method.
199+
- The third value ``false`` is a flag that indicates whether or not the model also applies to all overrides of the method.
164200
- The fourth value ``getInputStream`` is the method name.
165201
- The fifth value ``()`` is the method input type signature.
166202

@@ -185,15 +221,22 @@ This pattern covers many of the cases where we need to summarize flow through a
185221
186222
We need to add tuples to the ``summaryModel(package, type, subtypes, name, signature, ext, input, output, kind, provenance)`` extensible predicate by updating a data extension file:
187223

188-
.. code-block:: yaml
189-
190-
extensions:
191-
- addsTo:
192-
pack: codeql/java-all
193-
extensible: summaryModel
194-
data:
195-
- ["java.lang", "String", False, "concat", "(String)", "", "Argument[this]", "ReturnValue", "taint", "manual"]
196-
- ["java.lang", "String", False, "concat", "(String)", "", "Argument[0]", "ReturnValue", "taint", "manual"]
224+
.. code-block:: json
225+
226+
{
227+
"extensions": [
228+
{
229+
"addsTo": {
230+
"pack": "codeql/java-all",
231+
"extensible": "summaryModel"
232+
},
233+
"data": [
234+
["java.lang", "String", false, "concat", "(String)", "", "Argument[this]", "ReturnValue", "taint", "manual"],
235+
["java.lang", "String", false, "concat", "(String)", "", "Argument[0]", "ReturnValue", "taint", "manual"]
236+
]
237+
}
238+
]
239+
}
197240
198241
Each tuple defines flow from one argument to the return value.
199242
The first row defines flow from the qualifier (``s1`` in the example) to the return value (``t`` in the example) and the second row defines flow from the first argument (``s2`` in the example) to the return value (``t`` in the example).
@@ -203,7 +246,7 @@ These are the same for both of the rows above as we are adding two summaries for
203246

204247
- The first value ``java.lang`` is the package name.
205248
- The second value ``String`` is the class (type) name.
206-
- The third value ``False`` is a flag that indicates whether or not the model also applies to all overrides of the method.
249+
- The third value ``false`` is a flag that indicates whether or not the model also applies to all overrides of the method.
207250
- The fourth value ``concat`` is the method name.
208251
- The fifth value ``(String)`` is the method input type signature.
209252

@@ -229,23 +272,30 @@ Here we model flow through higher order methods and collection types.
229272
230273
We need to add tuples to the ``summaryModel(package, type, subtypes, name, signature, ext, input, output, kind, provenance)`` extensible predicate by updating a data extension file:
231274

232-
.. code-block:: yaml
233-
234-
extensions:
235-
- addsTo:
236-
pack: codeql/java-all
237-
extensible: summaryModel
238-
data:
239-
- ["java.util.stream", "Stream", True, "map", "(Function)", "", "Argument[this].Element", "Argument[0].Parameter[0]", "value", "manual"]
240-
- ["java.util.stream", "Stream", True, "map", "(Function)", "", "Argument[0].ReturnValue", "ReturnValue.Element", "value", "manual"]
275+
.. code-block:: json
276+
277+
{
278+
"extensions": [
279+
{
280+
"addsTo": {
281+
"pack": "codeql/java-all",
282+
"extensible": "summaryModel"
283+
},
284+
"data": [
285+
["java.util.stream", "Stream", true, "map", "(Function)", "", "Argument[this].Element", "Argument[0].Parameter[0]", "value", "manual"],
286+
["java.util.stream", "Stream", true, "map", "(Function)", "", "Argument[0].ReturnValue", "ReturnValue.Element", "value", "manual"]
287+
]
288+
}
289+
]
290+
}
241291
242292
Each tuple defines part of the flow that comprises the total flow through the ``map`` method.
243293
The first five values identify the callable (in this case a method) to be modeled as a summary.
244294
These are the same for both of the rows above as we are adding two summaries for the same method.
245295

246296
- The first value ``java.util.stream`` is the package name.
247297
- The second value ``Stream`` is the class (type) name.
248-
- The third value ``True`` is a flag that indicates whether or not the model also applies to all overrides of the method.
298+
- The third value ``true`` is a flag that indicates whether or not the model also applies to all overrides of the method.
249299
- The fourth value ``map`` is the method name.
250300
- The fifth value ``Function`` is the method input type signature.
251301

@@ -286,20 +336,27 @@ This is the ``getName`` method in the ``File`` class, which is located in the ``
286336
287337
We need to add a tuple to the ``barrierModel(package, type, subtypes, name, signature, ext, output, kind, provenance)`` extensible predicate by updating a data extension file.
288338

289-
.. code-block:: yaml
290-
291-
extensions:
292-
- addsTo:
293-
pack: codeql/java-all
294-
extensible: barrierModel
295-
data:
296-
- ["java.io", "File", True, "getName", "()", "", "ReturnValue", "path-injection", "manual"]
339+
.. code-block:: json
340+
341+
{
342+
"extensions": [
343+
{
344+
"addsTo": {
345+
"pack": "codeql/java-all",
346+
"extensible": "barrierModel"
347+
},
348+
"data": [
349+
["java.io", "File", true, "getName", "()", "", "ReturnValue", "path-injection", "manual"]
350+
]
351+
}
352+
]
353+
}
297354
298355
The first five values identify the callable (in this case a method) to be modeled as a barrier.
299356

300357
- The first value ``java.io`` is the package name.
301358
- The second value ``File`` is the name of the class (type) that contains the method.
302-
- The third value ``True`` is a flag that indicates whether or not the model also applies to all overrides of the method.
359+
- The third value ``true`` is a flag that indicates whether or not the model also applies to all overrides of the method.
303360
- The fourth value ``getName`` is the method name.
304361
- The fifth value ``()`` is the method input type signature.
305362

@@ -328,20 +385,27 @@ When the ``isAbsolute`` method returns ``false``, the URI is relative and theref
328385
329386
We need to add a tuple to the ``barrierGuardModel(package, type, subtypes, name, signature, ext, input, acceptingValue, kind, provenance)`` extensible predicate by updating a data extension file.
330387

331-
.. code-block:: yaml
332-
333-
extensions:
334-
- addsTo:
335-
pack: codeql/java-all
336-
extensible: barrierGuardModel
337-
data:
338-
- ["java.net", "URI", True, "isAbsolute", "()", "", "Argument[this]", "false", "request-forgery", "manual"]
388+
.. code-block:: json
389+
390+
{
391+
"extensions": [
392+
{
393+
"addsTo": {
394+
"pack": "codeql/java-all",
395+
"extensible": "barrierGuardModel"
396+
},
397+
"data": [
398+
["java.net", "URI", true, "isAbsolute", "()", "", "Argument[this]", "false", "request-forgery", "manual"]
399+
]
400+
}
401+
]
402+
}
339403
340404
The first five values identify the callable (in this case a method) to be modeled as a barrier guard.
341405

342406
- The first value ``java.net`` is the package name.
343407
- The second value ``URI`` is the name of the class (type) that contains the method.
344-
- The third value ``True`` is a flag that indicates whether or not the model guard also applies to all overrides of the method.
408+
- The third value ``true`` is a flag that indicates whether or not the model guard also applies to all overrides of the method.
345409
- The fourth value ``isAbsolute`` is the method name.
346410
- The fifth value ``()`` is the method input type signature.
347411

@@ -367,14 +431,21 @@ A neutral model is used to define that there is no flow through a method.
367431
368432
We need to add a tuple to the ``neutralModel(package, type, name, signature, kind, provenance)`` extensible predicate by updating a data extension file.
369433

370-
.. code-block:: yaml
371-
372-
extensions:
373-
- addsTo:
374-
pack: codeql/java-all
375-
extensible: neutralModel
376-
data:
377-
- ["java.time", "Instant", "now", "()", "summary", "manual"]
434+
.. code-block:: json
435+
436+
{
437+
"extensions": [
438+
{
439+
"addsTo": {
440+
"pack": "codeql/java-all",
441+
"extensible": "neutralModel"
442+
},
443+
"data": [
444+
["java.time", "Instant", "now", "()", "summary", "manual"]
445+
]
446+
}
447+
]
448+
}
378449
379450
The first four values identify the callable (in this case a method) to be modeled as a neutral, the fifth value is the kind, and the sixth value is the provenance (origin) of the neutral.
380451

0 commit comments

Comments
 (0)