feat(lang): add dependency origin graph (#1970)

Co-authored-by: knqyf263 <knqyf263@gmail.com>
This commit is contained in:
AndreyLevchenko
2022-06-16 13:34:26 +06:00
committed by GitHub
parent 685a92e09a
commit 3e3c119555
14 changed files with 314 additions and 25 deletions

View File

@@ -6,6 +6,90 @@
$ trivy image -f table golang:1.12-alpine
```
### Show origins of vulnerable dependencies
!!! warning "EXPERIMENTAL"
This feature might change without preserving backwards compatibility.
Modern software development relies on the use of third-party libraries.
Third-party dependencies also depend on others so a list of dependencies can be represented as a dependency graph.
In some cases, vulnerable dependencies are not linked directly, and it requires analyses of the tree.
To make this task simpler Trivy can show a dependency origin tree with the `--dependency-tree` flag.
This flag is available with the `--format table` flag only.
This tree is the reverse of the npm list command.
However, if you want to resolve a vulnerability in a particular indirect dependency, the reversed tree is useful to know where that dependency comes from and identify which package you actually need to update.
In table output, it looks like:
```sh
$ trivy fs --severity HIGH,CRITICAL --dependency-tree /path/to/your_node_project
package-lock.json (npm)
=======================
Total: 2 (HIGH: 1, CRITICAL: 1)
┌──────────────────┬────────────────┬──────────┬───────────────────┬───────────────┬────────────────────────────────────────────────────────────┐
│ Library │ Vulnerability │ Severity │ Installed Version │ Fixed Version │ Title │
├──────────────────┼────────────────┼──────────┼───────────────────┼───────────────┼────────────────────────────────────────────────────────────┤
│ follow-redirects │ CVE-2022-0155 │ HIGH │ 1.14.6 │ 1.14.7 │ follow-redirects: Exposure of Private Personal Information │
│ │ │ │ │ │ to an Unauthorized Actor │
│ │ │ │ │ │ https://avd.aquasec.com/nvd/cve-2022-0155 │
├──────────────────┼────────────────┼──────────┼───────────────────┼───────────────┼────────────────────────────────────────────────────────────┤
│ glob-parent │ CVE-2020-28469 │ CRITICAL │ 3.1.0 │ 5.1.2 │ nodejs-glob-parent: Regular expression denial of service │
│ │ │ │ │ │ https://avd.aquasec.com/nvd/cve-2020-28469 │
└──────────────────┴────────────────┴──────────┴───────────────────┴───────────────┴────────────────────────────────────────────────────────────┘
Dependency Origin Tree
======================
package-lock.json
├── follow-redirects@1.14.6, (HIGH: 1, CRITICAL: 0)
│ └── axios@0.21.4
└── glob-parent@3.1.0, (HIGH: 0, CRITICAL: 1)
└── chokidar@2.1.8
└── watchpack-chokidar2@2.0.1
└── watchpack@1.7.5
└── webpack@4.46.0
└── cra-append-sw@2.7.0
```
Vulnerable dependencies are shown in the top level of the tree.
Lower levels show how those vulnerabilities are introduced.
In the example above **axios@0.21.4** included in the project directly depends on the vulnerable **follow-redirects@1.14.6**.
Also, **glob-parent@3.1.0** with some vulnerabilities is included through chain of dependencies that is added by **cra-append-sw@2.7.0**.
Then, you can try to update **axios@0.21.4** and **cra-append-sw@2.7.0** to resolve vulnerabilities in **follow-redirects@1.14.6** and **glob-parent@3.1.0**.
!!! note
Only Node.js (package-lock.json) is supported at the moment.
## JSON
Similar structure is included in JSON output format
```json
"VulnerabilityID": "CVE-2022-0235",
"PkgID": "node-fetch@1.7.3",
"PkgName": "node-fetch",
"PkgParents": [
{
"ID": "isomorphic-fetch@2.2.1",
"Parents": [
{
"ID": "fbjs@0.8.18",
"Parents": [
{
"ID": "styled-components@3.1.3"
}
]
}
]
}
],
```
!!! caution
As of May 2022 the feature is supported for `npm` dependency parser only
## JSON
```

30
go.mod
View File

@@ -7,6 +7,7 @@ require (
github.com/Masterminds/sprig/v3 v3.2.2
github.com/NYTimes/gziphandler v1.1.1
github.com/aquasecurity/bolt-fixtures v0.0.0-20200903104109-d34e7f983986
github.com/aquasecurity/fanal v0.0.0-20220615115521-e411bc995c6d
github.com/aquasecurity/go-dep-parser v0.0.0-20220607141748-ab2deea55bdf
github.com/aquasecurity/go-gem-version v0.0.0-20201115065557-8eed6fe000ce
github.com/aquasecurity/go-npm-version v0.0.0-20201110091526-0b796d180798
@@ -42,6 +43,7 @@ require (
github.com/tetratelabs/wazero v0.0.0-20220606011721-119b069ba23e
github.com/twitchtv/twirp v8.1.2+incompatible
github.com/urfave/cli/v2 v2.8.1
github.com/xlab/treeprint v1.1.0
go.uber.org/zap v1.21.0
golang.org/x/exp v0.0.0-20220407100705-7b9b53b0aca4
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1
@@ -50,8 +52,6 @@ require (
k8s.io/utils v0.0.0-20220210201930-3a6ce19ff2f9
)
require github.com/aquasecurity/fanal v0.0.0-20220615115521-e411bc995c6d
require (
cloud.google.com/go v0.99.0 // indirect
cloud.google.com/go/storage v1.14.0 // indirect
@@ -257,7 +257,6 @@ require (
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
github.com/xeipuuv/gojsonschema v1.2.0 // indirect
github.com/xlab/treeprint v0.0.0-20181112141820-a009c3971eca // indirect
github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673 // indirect
github.com/yashtewari/glob-intersection v0.1.0 // indirect
github.com/zclconf/go-cty v1.10.0 // indirect
@@ -284,9 +283,22 @@ require (
google.golang.org/grpc v1.47.0 // indirect
gopkg.in/cheggaaa/pb.v1 v1.0.28 // indirect
gopkg.in/go-playground/validator.v9 v9.31.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
lukechampine.com/uint128 v1.1.1 // indirect
modernc.org/cc/v3 v3.36.0 // indirect
modernc.org/ccgo/v3 v3.16.6 // indirect
modernc.org/libc v1.16.7 // indirect
modernc.org/mathutil v1.4.1 // indirect
modernc.org/memory v1.1.1 // indirect
modernc.org/opt v0.1.1 // indirect
modernc.org/sqlite v1.17.3 // indirect
modernc.org/strutil v1.1.1 // indirect
modernc.org/token v1.0.0 // indirect
)
require (
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/square/go-jose.v2 v2.5.1 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gotest.tools v2.2.0+incompatible
helm.sh/helm/v3 v3.9.0 // indirect
@@ -300,16 +312,6 @@ require (
k8s.io/klog/v2 v2.60.1 // indirect
k8s.io/kube-openapi v0.0.0-20220328201542-3ee0da9b0b42 // indirect
k8s.io/kubectl v0.24.1 // indirect
lukechampine.com/uint128 v1.1.1 // indirect
modernc.org/cc/v3 v3.36.0 // indirect
modernc.org/ccgo/v3 v3.16.6 // indirect
modernc.org/libc v1.16.7 // indirect
modernc.org/mathutil v1.4.1 // indirect
modernc.org/memory v1.1.1 // indirect
modernc.org/opt v0.1.1 // indirect
modernc.org/sqlite v1.17.3 // indirect
modernc.org/strutil v1.1.1 // indirect
modernc.org/token v1.0.0 // indirect
oras.land/oras-go v1.1.1 // indirect
sigs.k8s.io/json v0.0.0-20211208200746-9f7c6b3444d2 // indirect
sigs.k8s.io/kustomize/api v0.11.4 // indirect

3
go.sum
View File

@@ -1252,8 +1252,9 @@ github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415/go.mod h1:
github.com/xeipuuv/gojsonschema v1.2.0 h1:LhYJRs+L4fBtjZUfuSZIKGeVu0QRy8e5Xi7D17UxZ74=
github.com/xeipuuv/gojsonschema v1.2.0/go.mod h1:anYRn/JVcOK2ZgGU+IjEV4nwlhoK5sQluxsYJ78Id3Y=
github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=
github.com/xlab/treeprint v0.0.0-20181112141820-a009c3971eca h1:1CFlNzQhALwjS9mBAUkycX616GzgsuYUOCHA5+HSlXI=
github.com/xlab/treeprint v0.0.0-20181112141820-a009c3971eca/go.mod h1:ce1O1j6UtZfjr22oyGxGLbauSBp2YVXpARAosm7dHBg=
github.com/xlab/treeprint v1.1.0 h1:G/1DjNkPpfZCFt9CSh6b5/nY4VimlbHF3Rh4obvtzDk=
github.com/xlab/treeprint v1.1.0/go.mod h1:gj5Gd3gPdKtR1ikdDK6fnFLdmIS0X30kTTuNd/WEJu0=
github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673 h1:bAn7/zixMGCfxrRTfdpNzjtPYqr8smhKouy9mxVdGPU=
github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673/go.mod h1:N3UwUGtsrSj3ccvlPHLoLsHnpR27oXr4ZE984MbSER8=

View File

@@ -22,6 +22,7 @@
"Vulnerabilities": [
{
"VulnerabilityID": "CVE-2019-11358",
"PkgID": "jquery@3.3.9",
"PkgName": "jquery",
"InstalledVersion": "3.3.9",
"FixedVersion": "3.4.0",
@@ -137,6 +138,7 @@
},
{
"VulnerabilityID": "CVE-2019-10744",
"PkgID": "lodash@4.17.4",
"PkgName": "lodash",
"InstalledVersion": "4.17.4",
"FixedVersion": "4.17.12",

View File

@@ -371,6 +371,12 @@ var (
EnvVars: []string{"TRIVY_SECRET_CONFIG"},
}
dependencyTree = cli.BoolFlag{
Name: "dependency-tree",
Usage: "show dependency origin tree (EXPERIMENTAL)",
EnvVars: []string{"TRIVY_DEPENDENCY_TREE"},
}
// Global flags
globalFlags = []cli.Flag{
&quietFlag,
@@ -499,6 +505,7 @@ func NewImageCommand() *cli.Command {
&insecureFlag,
&dbRepositoryFlag,
&secretConfig,
&dependencyTree,
stringSliceFlag(skipFiles),
stringSliceFlag(skipDirs),
@@ -545,6 +552,7 @@ func NewFilesystemCommand() *cli.Command {
&offlineScan,
&dbRepositoryFlag,
&secretConfig,
&dependencyTree,
stringSliceFlag(skipFiles),
stringSliceFlag(skipDirs),
@@ -595,6 +603,7 @@ func NewRootfsCommand() *cli.Command {
&offlineScan,
&dbRepositoryFlag,
&secretConfig,
&dependencyTree,
stringSliceFlag(skipFiles),
stringSliceFlag(skipDirs),
stringSliceFlag(configPolicy),
@@ -641,6 +650,7 @@ func NewRepositoryCommand() *cli.Command {
&insecureFlag,
&dbRepositoryFlag,
&secretConfig,
&dependencyTree,
stringSliceFlag(skipFiles),
stringSliceFlag(skipDirs),
},
@@ -681,6 +691,7 @@ func NewClientCommand() *cli.Command {
&offlineScan,
&insecureFlag,
&secretConfig,
&dependencyTree,
&token,
&tokenHeader,

View File

@@ -243,6 +243,7 @@ func (r *runner) Report(opt Option, report types.Report) error {
AppVersion: opt.GlobalOption.AppVersion,
Format: opt.Format,
Output: opt.Output,
Tree: opt.DependencyTree,
Severities: opt.Severities,
OutputTemplate: opt.Template,
IncludeNonFailures: opt.IncludeNonFailures,

View File

@@ -18,6 +18,7 @@ import (
type ReportOption struct {
Format string
Template string
DependencyTree bool
IgnoreFile string
IgnoreUnfixed bool
@@ -43,6 +44,7 @@ func NewReportOption(c *cli.Context) ReportOption {
return ReportOption{
output: c.String("output"),
Format: c.String("format"),
DependencyTree: c.Bool("dependency-tree"),
Template: c.String("template"),
IgnorePolicy: c.String("ignore-policy"),
@@ -76,6 +78,11 @@ func (c *ReportOption) Init(output io.Writer, logger *zap.SugaredLogger) error {
logger.Warn(`"--list-all-pkgs" cannot be used with "--format table". Try "--format json" or other formats.`)
}
// "--dependency-tree" option is available only with "--format table".
if c.DependencyTree && c.Format != "table" {
logger.Warn(`"--dependency-tree" can be used only with "--format table".`)
}
if c.forceListAllPkgs(logger) {
c.ListAllPkgs = true
}
@@ -141,6 +148,10 @@ func (c *ReportOption) forceListAllPkgs(logger *zap.SugaredLogger) bool {
logger.Debugf("'github', 'cyclonedx', 'spdx', and 'spdx-json' automatically enables '--list-all-pkgs'.")
return true
}
if c.DependencyTree {
logger.Debugf("'--dependency-tree' enables '--list-all-pkgs'.")
return true
}
return false
}

View File

@@ -25,7 +25,7 @@ func Detect(libType string, pkgs []ftypes.Package) ([]types.DetectedVulnerabilit
func detect(driver Driver, libs []ftypes.Package) ([]types.DetectedVulnerability, error) {
var vulnerabilities []types.DetectedVulnerability
for _, lib := range libs {
vulns, err := driver.DetectVulnerabilities(lib.Name, lib.Version)
vulns, err := driver.DetectVulnerabilities(lib.ID, lib.Name, lib.Version)
if err != nil {
return nil, xerrors.Errorf("failed to detect %s vulnerabilities: %w", driver.Type(), err)
}

View File

@@ -75,7 +75,7 @@ func (d *Driver) Type() string {
// If "ecosystem" is pip, it looks for buckets with "pip::" and gets security advisories from those buckets.
// It allows us to add a new data source with the ecosystem prefix (e.g. pip::new-data-source)
// and detect vulnerabilities without specifying a specific bucket name.
func (d *Driver) DetectVulnerabilities(pkgName, pkgVer string) ([]types.DetectedVulnerability, error) {
func (d *Driver) DetectVulnerabilities(pkgID, pkgName, pkgVer string) ([]types.DetectedVulnerability, error) {
// e.g. "pip::", "npm::"
prefix := fmt.Sprintf("%s::", d.ecosystem)
advisories, err := d.dbc.GetAdvisories(prefix, vulnerability.NormalizePkgName(d.ecosystem, pkgName))
@@ -91,6 +91,7 @@ func (d *Driver) DetectVulnerabilities(pkgName, pkgVer string) ([]types.Detected
vuln := types.DetectedVulnerability{
VulnerabilityID: adv.VulnerabilityID,
PkgID: pkgID,
PkgName: pkgName,
InstalledVersion: pkgVer,
FixedVersion: createFixedVersions(adv),

View File

@@ -142,7 +142,7 @@ func TestDriver_Detect(t *testing.T) {
driver, err := library.NewDriver(tt.libType)
require.NoError(t, err)
got, err := driver.DetectVulnerabilities(tt.args.pkgName, tt.args.pkgVer)
got, err := driver.DetectVulnerabilities("", tt.args.pkgName, tt.args.pkgVer)
if tt.wantErr != "" {
require.Error(t, err)
assert.Contains(t, err.Error(), tt.wantErr)

View File

@@ -10,6 +10,8 @@ import (
"github.com/fatih/color"
"github.com/liamg/tml"
"github.com/samber/lo"
"github.com/xlab/treeprint"
"golang.org/x/exp/slices"
ftypes "github.com/aquasecurity/fanal/types"
@@ -34,6 +36,9 @@ type TableWriter struct {
Severities []dbTypes.Severity
Output io.Writer
// Show dependency origin tree
Tree bool
// We have to show a message once about using the '-format json' subcommand to get the full pkgPath
ShowMessageOnce *sync.Once
@@ -122,6 +127,10 @@ func (tw TableWriter) write(result types.Result) {
_, _ = fmt.Fprint(tw.Output, NewMisconfigRenderer(result.Target, result.Misconfigurations, tw.IncludeNonFailures, tw.isOutputToTerminal()).Render())
}
if tw.Tree {
tw.renderDependencyTree(result)
}
// For debugging
if tw.Trace {
tw.outputTrace(result)
@@ -196,6 +205,81 @@ func (tw TableWriter) setVulnerabilityRows(tableWriter *table.Table, vulns []typ
tableWriter.AddRow(row...)
}
}
func (tw TableWriter) renderDependencyTree(result types.Result) {
// Get parents of each dependency
parents := reverseDeps(result.Packages)
if len(parents) == 0 {
return
}
root := treeprint.NewWithRoot(fmt.Sprintf(`
Dependency Origin Tree
======================
%s`, result.Target))
// This count is next to the package ID.
// e.g. node-fetch@1.7.3 (MEDIUM: 2, HIGH: 1, CRITICAL: 3)
pkgSeverityCount := map[string]map[string]int{}
for _, vuln := range result.Vulnerabilities {
cnts, ok := pkgSeverityCount[vuln.PkgID]
if !ok {
cnts = map[string]int{}
}
cnts[vuln.Severity]++
pkgSeverityCount[vuln.PkgID] = cnts
}
// Render tree
seen := map[string]struct{}{}
for _, vuln := range result.Vulnerabilities {
if _, ok := seen[vuln.PkgID]; ok {
continue
}
_, summaries := tw.summary(pkgSeverityCount[vuln.PkgID])
topLvlID := fmt.Sprintf("%s, (%s)", vuln.PkgID, strings.Join(summaries, ", "))
if tw.isOutputToTerminal() {
topLvlID = color.HiRedString(topLvlID)
}
seen[vuln.PkgID] = struct{}{}
branch := root.AddBranch(topLvlID)
addParents(branch, vuln.PkgID, parents)
}
tw.Println(root.String())
}
func addParents(topItem treeprint.Tree, pkgID string, parentMap map[string][]string) {
parents, ok := parentMap[pkgID]
if !ok {
return
}
for _, parent := range parents {
branch := topItem.AddBranch(parent)
addParents(branch, parent, parentMap)
}
}
func reverseDeps(libs []ftypes.Package) map[string][]string {
reversed := make(map[string][]string)
for _, lib := range libs {
for _, dependOn := range lib.DependsOn {
items, ok := reversed[dependOn]
if !ok {
reversed[dependOn] = []string{lib.ID}
} else {
reversed[dependOn] = append(items, lib.ID)
}
}
}
for k, v := range reversed {
reversed[k] = lo.Uniq(v)
}
return reversed
}
func (tw TableWriter) outputTrace(result types.Result) {
blue := color.New(color.FgBlue).SprintFunc()

View File

@@ -6,6 +6,7 @@ import (
"github.com/stretchr/testify/assert"
ftypes "github.com/aquasecurity/fanal/types"
dbTypes "github.com/aquasecurity/trivy-db/pkg/types"
"github.com/aquasecurity/trivy/pkg/report"
"github.com/aquasecurity/trivy/pkg/types"
@@ -136,6 +137,91 @@ func TestReportWriter_Table(t *testing.T) {
name: "no vulns",
expectedOutput: ``,
},
{
name: "happy path with vulnerability origin graph",
results: types.Results{
{
Target: "package-lock.json",
Class: "lang-pkgs",
Type: "npm",
Packages: []ftypes.Package{
{
ID: "node-fetch@1.7.3",
Name: "node-fetch",
Version: "1.7.3",
},
{
ID: "isomorphic-fetch@2.2.1",
Name: "isomorphic-fetch",
Version: "2.2.1",
DependsOn: []string{
"node-fetch@1.7.3",
},
},
{
ID: "fbjs@0.8.18",
Name: "fbjs",
Version: "0.8.18",
DependsOn: []string{
"isomorphic-fetch@2.2.1",
},
},
{
ID: "styled-components@3.1.3",
Name: "styled-components",
Version: "3.1.3",
DependsOn: []string{
"fbjs@0.8.18",
},
},
},
Vulnerabilities: []types.DetectedVulnerability{
{
VulnerabilityID: "CVE-2022-0235",
PkgID: "node-fetch@1.7.3",
PkgName: "node-fetch",
Vulnerability: dbTypes.Vulnerability{
Title: "foobar",
Description: "baz",
Severity: "HIGH",
},
InstalledVersion: "1.7.3",
FixedVersion: "2.6.7, 3.1.1",
},
{
VulnerabilityID: "CVE-2021-26539",
PkgID: "sanitize-html@1.20.0",
PkgName: "sanitize-html",
Vulnerability: dbTypes.Vulnerability{
Title: "foobar",
Description: "baz",
Severity: "MEDIUM",
},
InstalledVersion: "1.20.0",
FixedVersion: "2.3.1",
},
},
},
},
expectedOutput: `┌───────────────┬────────────────┬──────────┬───────────────────┬───────────────┬────────┐
│ Library │ Vulnerability │ Severity │ Installed Version │ Fixed Version │ Title │
├───────────────┼────────────────┼──────────┼───────────────────┼───────────────┼────────┤
│ node-fetch │ CVE-2022-0235 │ HIGH │ 1.7.3 │ 2.6.7, 3.1.1 │ foobar │
├───────────────┼────────────────┼──────────┼───────────────────┼───────────────┤ │
│ sanitize-html │ CVE-2021-26539 │ MEDIUM │ 1.20.0 │ 2.3.1 │ │
└───────────────┴────────────────┴──────────┴───────────────────┴───────────────┴────────┘
Dependency Origin Tree
======================
package-lock.json
├── node-fetch@1.7.3, (MEDIUM: 0, HIGH: 1)
│ └── isomorphic-fetch@2.2.1
│ └── fbjs@0.8.18
│ └── styled-components@3.1.3
└── sanitize-html@1.20.0, (MEDIUM: 1, HIGH: 0)
`,
},
}
for _, tc := range testCases {
@@ -144,7 +230,9 @@ func TestReportWriter_Table(t *testing.T) {
err := report.Write(types.Report{Results: tc.results}, report.Option{
Format: "table",
Output: &tableWritten,
Tree: true,
IncludeNonFailures: tc.includeNonFailures,
Severities: []dbTypes.Severity{dbTypes.SeverityHigh, dbTypes.SeverityMedium},
})
assert.NoError(t, err)
assert.Equal(t, tc.expectedOutput, tableWritten.String(), tc.name)

View File

@@ -29,11 +29,13 @@ const (
)
type Option struct {
AppVersion string
Format string
Output io.Writer
Tree bool
Severities []dbTypes.Severity
OutputTemplate string
AppVersion string
// For misconfigurations
IncludeNonFailures bool
@@ -48,6 +50,7 @@ func Write(report types.Report, option Option) error {
writer = &TableWriter{
Output: option.Output,
Severities: option.Severities,
Tree: option.Tree,
ShowMessageOnce: &sync.Once{},
IncludeNonFailures: option.IncludeNonFailures,
Trace: option.Trace,

View File

@@ -9,6 +9,7 @@ import (
type DetectedVulnerability struct {
VulnerabilityID string `json:",omitempty"`
VendorIDs []string `json:",omitempty"`
PkgID string `json:",omitempty"` // It is used to construct dependency graph.
PkgName string `json:",omitempty"`
PkgPath string `json:",omitempty"` // It will be filled in the case of language-specific packages such as egg/wheel and gemspec
InstalledVersion string `json:",omitempty"`