Commit Graph

82 Commits

Author SHA1 Message Date
Andrew Eisenberg
fa2bc211fd Merge branch 'aeisenberg/unrevert-query-filters' into aeisenberg/fix-config-files 2022-08-11 14:57:16 -07:00
Andrew Eisenberg
072cd929a3 Merge remote-tracking branch 'upstream/main' into aeisenberg/unrevert-query-filters 2022-08-11 10:00:12 -07:00
Andrew Eisenberg
d74f663ed4 Merge remote-tracking branch 'upstream/main' into aeisenberg/fix-config-files 2022-08-11 09:57:45 -07:00
Andrew Eisenberg
0403fb7d8c Merge branch 'main' into aeisenberg/fix-config-files 2022-08-10 15:39:35 -07:00
Andrew Eisenberg
44f42da9ca Merge branch 'main' into aeisenberg/unrevert-query-filters 2022-08-10 15:22:40 -07:00
Angela P Wen
90676d9cb9 Merge remote-tracking branch 'origin/main' into angelapwen/post-init-cleanup 2022-08-10 15:08:04 +02:00
Edoardo Pirovano
8f867dcb21 Introduce TRAP caching 2022-08-05 17:48:05 +01:00
Angela P Wen
2746051310 Catch case where database isn't finalized 2022-07-29 12:00:07 +02:00
Andrew Eisenberg
01d16b1e01 Merge branch 'main' into aeisenberg/fix-config-files 2022-07-13 14:05:48 -07:00
Henry Mercer
dca60ba711 Merge remote-tracking branch 'origin/main' into henrymercer/one-click-debug 2022-07-12 16:54:30 +01:00
Cornelius Riemenschneider
3dcdbc9add Unset tracing variables after finalizing databases.
The tracer is very good at preserving itself, so unsetting the tracing-specific
variables from within a process will not end tracing for children of
that process.
The way the actions process model works means that we're running inside
a process for the entire build step that was launched with the tracer
variables set, so we'll have the tracer injected into the entire build
step and its children.
If we unset the variables in end-tracing, we will get into an intermediate
state: Not all variables in there are preserved by the tracer,
but the tracer is still active.
Usually, that wouldn't be a problem, but the autobuilders called from
the finalize step will suddenly run under a half-configured tracer.

Particularly, this half-configured tracer is unable to execute the dotnet
CLI without hangs, as the environment variable that prevents hangs for
dotnet on MacOS has been unset, but the tracer is still active.

This is an issue for the the go autobuilder, that invokes
user-provided build scripts in the hope of installing dependencies.
If that build script then invokes dotnet, it will hang.

This is only of concern for the Lua tracer that now implements proper
multi-language tracing: Previously, when encountering the go autobuilder,
the tracer disabled itself entirely, thus side-stepping any hangs.
In the new, multi-language tracing world, the tracer will stay active
as long as there is at least one other language that's been set up
for tracing.
Thus, we also get hangs when invoking the dotnet CLI through the go
autobuilder.
2022-07-12 11:33:44 +00:00
Henry Mercer
7e94a6cbca Print diagnostic messages when debug logging enabled
This commit prints diagnostic messages to the Actions log when debug
logging is enabled by passing `debug: true` to `codeql-action/init` or
enabling Actions step debug logging.
2022-07-01 18:56:12 +01:00
Henry Mercer
53850d88bb Enable one-click debugging via the "Enable debug logging" rerun option 2022-07-01 17:56:57 +01:00
Andrew Eisenberg
8688a09e14 When using codescanning config call run queries a single time
When the codescanning config is being used by the CLI, there is a
single query suite that is generated that contains all queries to be
run by the analysis. This is different from the traditional way, where
there are potentially three query suites: builtin, custom, and packs.

We need to ensure that when the codescanning config is being used,
only a single call to run queries is used, and this call uses the
single generated query suite.

Also, this commit changes the cutoff version for codescanning config to
2.10.1. Earlier versions work, but there were some bugs that are only
fixed in 2.10.1 and later.
2022-06-29 12:50:24 -07:00
Andrew Eisenberg
6fabde2be8 Add packs and queries from input
This commit adds the packs and queries from the actions input to the
config file used by the CodeQL CLI.

When the `+` is used, the actions input value is combined with the
config value and when it is not used, the input value overrides the
config value.

This commit also adds a bunch of integration tests for this feature.
In order to avoid adding too many new jobs, all of the tests are
run sequentially in a single job (matrixed across relevant operating
systems and OSes).
2022-06-28 14:07:51 -07:00
Andrew Eisenberg
237260b693 Revert "Revert usage of --codescanning-config flag"
This reverts commit 43d066495c.
2022-06-28 13:03:04 -07:00
Andrew Eisenberg
2a70419420 Revert "Revert "Add capability to filter queries #1098""
This reverts commit 99d4397d88.
2022-06-27 13:13:55 -07:00
Cornelius Riemenschneider
ab7316e0c5 Implement unit tests for reading the Lua tracer FF in analyze-action. 2022-06-27 16:04:29 +00:00
Cornelius Riemenschneider
f422a50448 Honor the Lua tracer FF for database trace-command invocations for scanned languages.
In theory, a scanned language will not setup the build tracer, and so
shouldn't care about lua versus legacy tracing. However, `go` is a
special case where the autobuilder runs under the build tracer, that
then gets disabled immediately again, unless a special environment
variable is used.
Therefore, we need to thread through the feature flag to this
`database trace-command` invocation. For other scanned languages,
this should be a no-op, as no tracing is ever set up.
2022-06-27 16:04:29 +00:00
Andrew Eisenberg
99d4397d88 Revert "Add capability to filter queries #1098"
https://github.com/github/codeql-action/pull/1098
This reverts commit 777b778409.
This reverts commit 59ca9b59cb.
This reverts commit eec34d5f05.
This reverts commit 40b280032c.
2022-06-21 13:49:33 -07:00
Andrew Eisenberg
80ecdcdf69 Merge pull request #1098 from github/aeisenberg/remove-queries
Add capability to filter queries
2022-06-15 17:52:46 -07:00
Andrew Eisenberg
40b280032c Add capability to filter queries
This change adds a `query-filters` property to the codeql-config file.

This property is an array of `exclude`/`include` entries for a query
suite. These filters are appended to the generated query suite files
and used to filter queries after they are selected.

A related change is that now, all pack references are run in a single
query suite, which has the query filters appended to them.
2022-06-14 12:07:49 -07:00
Andrew Eisenberg
f7c46e5cbc Avoid use of rmdir
This is a deprecated method on node v16.
2022-06-13 22:40:09 +00:00
Andrew Eisenberg
922dc2b976 Use the --resolve-query-specs parameter of pack download
This will allow the command to resolve packs with paths.

Also, use a more concise version of `tr`.
2022-04-29 10:54:01 -07:00
Andrew Eisenberg
06b15c22b1 Allow pack specifiers to include paths
Also, this cleans up our pack-related integration tests.
We are now testing with the most recent CLIs.
2022-04-28 17:14:30 -07:00
Edoardo Pirovano
43d066495c Revert usage of --codescanning-config flag 2022-04-05 09:41:07 +01:00
Edoardo Pirovano
85cfdb24f4 Don't download packs when it isn't needed 2022-03-25 11:26:13 +00:00
Chris Gavin
938e0a0743 Add retryDelay and a type cast to make it work. 2022-02-28 19:42:58 +00:00
Chris Gavin
d63f798314 Retry directory deletion to avoid Windows file locking issues.
Co-authored-by: Andrew Eisenberg <aeisenberg@github.com>
2022-02-28 18:55:45 +00:00
Chris Gavin
86940df49f Delete the results directory in between runs. 2022-02-28 18:55:45 +00:00
Edoardo Pirovano
c592525a67 Respect end-tracing script instead of deleting one variable 2022-02-23 16:21:35 +00:00
Andrew Eisenberg
f8c38c1af3 Update changelog 2022-01-24 09:54:17 -08:00
Andrew Eisenberg
e6bcd71529 Remove experimental warning message for custom packs 2022-01-24 09:40:46 -08:00
Edoardo Pirovano
f04acbbdc3 Use CLI's own baseline LOC counting 2021-09-29 11:17:41 +01:00
Andrew Eisenberg
40568daca8 Fix compile errors introduced by typescript 4.4.2
4.4.2 introduces a breaking change that the variable in a catch clause
is now `unknown` type. So, we need to cast the `e`, `err`, or `error`
variables to type `Error`.
2021-09-10 14:06:27 -07:00
Edoardo Pirovano
2e71e02553 Remove old baseline LoC injection 2021-08-26 13:46:22 +01:00
Edoardo Pirovano
70d2efc353 Enforce a minimum version of CodeQL CLI 2021-08-18 11:48:12 +01:00
Edoardo Pirovano
0c4306b672 Pass --ram flag to database finalize 2021-08-12 18:12:28 +01:00
Edoardo Pirovano
d9849b8ca1 Rebuild after TypeScript version bump 2021-07-27 17:59:59 +01:00
Edoardo Pirovano
11d56696ec Augment where we insert baseline lines of code 2021-07-21 14:10:37 +01:00
Edoardo Pirovano
ef852c006a Support splitting of DB creation and query execution 2021-06-28 17:14:22 +01:00
Andrew Eisenberg
20aafcd90c Remove unnecessary type assertions 2021-06-10 11:15:03 -07:00
Andrew Eisenberg
23cdd6bab6 Remove two log entries
Cleans up the output logs even more. These entries don't add any more
information since everything is grouped now.
2021-06-09 14:32:08 -07:00
Andrew Eisenberg
a2e96a4c78 Add pack download to its own log group
Also, make the baseline count message less awkward sounding.
2021-06-09 14:13:05 -07:00
Andrew Eisenberg
82388fd94a Merge remote-tracking branch 'upstream/main' into aeisenberg/pack-run 2021-06-09 12:43:17 -07:00
Edoardo Pirovano
06df98a513 Add new log grouping for DB cleanup 2021-06-09 16:10:22 +01:00
Edoardo Pirovano
2cc885d66e Replace analyze with run-queries and interpret-results 2021-06-08 09:25:17 +01:00
Andrew Eisenberg
d87945e9fd Run the pack download command 2021-06-06 09:27:52 -07:00
Andrew Eisenberg
06687e95c8 Avoid using SemVer instances
Use strings instead. They are easier to serialize and deserialize.
2021-06-04 13:34:55 -07:00
Andrew Eisenberg
9b5753ab00 Fix logic for calculating if there are queries to run
During the analyze phase.
2021-06-04 13:23:35 -07:00