Velociraptor Artifact YAML Format & Packaging — Comprehensive Research

velociraptor-artifact-research · 3047 words

1. Artifact YAML Structure (Complete Schema)

Required Fields

  • name (string, required) — Unique identifier. Only [A-Za-z0-9_.] allowed; each dot-separated segment must start with a letter. Convention: dot-separated hierarchy (e.g., Windows.Applications.Chrome.History).

Top-Level Fields

Field Type Description Searchable
name string Unique identifier (required) Yes
aliases sequence Alternative names for the same artifact Yes
type string CLIENT (default), CLIENT_EVENT, SERVER, SERVER_EVENT, INTERNAL, NOTEBOOK Filterable
description string (markdown) Human-readable purpose & usage. Supports CommonMark. Start with a lead sentence. Yes
author string Author name/handle/email No
reference sequence List of URLs or markdown links for further reading No
parameters sequence User-definable parameters (see §7) No
sources sequence VQL query definitions producing result tables (see §6) No
precondition SELECT query Top-level OS/platform guard (evaluated before all sources)
export string (VQL) Shared VQL (LET statements) available to all sources and importing artifacts
imports sequence List of artifact names whose export sections to import
reports sequence Report templates (Go templates with VQL) for GUI rendering
column_types sequence Column display hints (e.g., timestamp, tree, preview_upload)
resources object Resource limits (timeout, max_rows, max_upload_bytes, ops_per_second)
tools sequence External binary tool definitions

Minimal Viable Artifact

name: Custom.Artifact.Name
description: |
  One-sentence summary of what this artifact does.
  
  Additional details here.

sources:
  - query: |
      SELECT * FROM info()
      LIMIT 10

2. Multi-OS Artifacts (Platform Conditions)

Two Approaches

A. Single Artifact with Source-Level Preconditions (RECOMMENDED for related data)

name: Custom.AIToolEvidence

sources:
  - name: WindowsData
    precondition: SELECT OS FROM info() WHERE OS = 'windows'
    query: |
      SELECT * FROM glob(globs='C:\\Users\\*\\AppData\\...')

  - name: LinuxData
    precondition: SELECT OS FROM info() WHERE OS = 'linux'
    query: |
      SELECT * FROM glob(globs='/home/*/.config/...')

  - name: MacOSData
    precondition: SELECT OS FROM info() WHERE OS = 'darwin'
    query: |
      SELECT * FROM glob(globs='/Users/*/Library/Application Support/...')

Critical: Source-level preconditions trigger PARALLEL execution mode — each source gets its own scope. Variables from one source are NOT visible to others.

B. Separate Per-OS Artifacts with a Common Alias

# In each OS-specific artifact:
name: Windows.Applications.Chrome.History
precondition: SELECT OS From info() where OS = 'windows'
...

# Then create a unified alias in a Generic artifact:
name: Generic.Detection.Yara.Glob
aliases:
  - Windows.Detection.Yara.Glob
  - Linux.Detection.Yara.Glob
  - MacOS.Detection.Yara.Glob

Common OS Preconditions

# Windows only
precondition: SELECT OS FROM info() WHERE OS = 'windows'

# Linux only
precondition: SELECT OS FROM info() WHERE OS = 'linux'

# macOS only
precondition: SELECT OS FROM info() WHERE OS = 'darwin'

# Linux + macOS (not arm64)
precondition: SELECT OS FROM info() WHERE OS =~ 'linux|darwin' AND NOT Architecture = 'arm64'

# All platforms (no precondition needed, or)
precondition: SELECT TRUE FROM scope()

# Admin privilege check (any OS)
precondition: SELECT * FROM info() WHERE IsAdmin

Cross-Platform Path Handling

Use pathspec() for Windows drive letters and glob() with OS-conditional globs:

-- Windows paths with drive letter
LET RootPath <= pathspec(Path=Root, accessor=Accessor)

-- Use accessor 'auto' for cross-platform (auto-selects OS/file/ntfs)
FROM glob(globs=PathGlob, accessor=Accessor)

-- Parse user from path (cross-platform pattern)
parse_string_with_regex(regex="/Users/(?P<User>[^/]+)", string=OSPath).User
-- or on Windows:
parse_string_with_regex(regex="\\\\Users\\\\(?P<User>[^\\\\]+)", string=OSPath).User

Real-World Multi-OS Example: Generic.Client.Info

This built-in artifact uses source-level preconditions for OS-specific data:

  • Source LinuxInfo with precondition: SELECT OS From info() where OS = 'linux'
  • Source WindowsInfo with precondition: SELECT OS From info() where OS = 'windows'
  • Source BasicInformation (no precondition — runs everywhere)
  • Source Users (Windows-only precondition)

3. Artifact Packaging & Distribution

How Artifacts Are Stored

  • Built-in artifacts are compiled into the Velociraptor binary
  • Custom artifacts are stored as YAML in <datastore>/artifact_definitions/
  • Directory structure mirrors the dot-separated name hierarchy:
    artifact_definitions/
    └── Custom/
        ├── Artifact/
        │   └── Name.yaml
        └── Windows/
            └── LastUser.yaml
    

Distribution Methods

  1. GUI Import — Server → Artifact → Import (paste YAML or upload ZIP)
  2. CLI --definitions flag — Accepts a directory or ZIP file:
    velociraptor --definitions /path/to/artifacts/ gui
    velociraptor --definitions artifacts.zip gui
    
  3. Config-based loading:
    • autoexec.artifact_definitions — Inline YAML in config
    • Frontend.artifact_definitions_directory — Single directory
    • defaults.artifact_definitions_directories — List of directories
  4. VQL artifact_set() function — Create artifacts programmatically
  5. Artifact Exchange — Community repository at https://exchange.velocidex.com

Packaging a Custom Artifact for Distribution

  1. Save as .yaml file following naming convention
  2. Optionally bundle multiple artifacts into a ZIP
  3. Import via GUI or CLI --definitions flag
  4. Custom artifacts appear with a ★ icon in the GUI

Built-in vs Custom vs Compiled-in

  • Compiled-in: In the binary, cannot be edited
  • Built-in (from config/directories): Cannot be edited during runtime, require server restart
  • Custom (in datastore): Editable at runtime via GUI or VQL

4. Best Practices for Forensic Collection Artifacts

File Collection with Glob Patterns

# Example from Generic.Collectors.File
parameters:
  - name: collectionSpec
    type: csv
    default: |
      Glob
      Users\*\NTUser.dat

sources:
  - query: |
      LET RootPath <= pathspec(Path=Root, accessor=Accessor)
      LET specs = SELECT RootPath + Glob AS Glob
        FROM collectionSpec
      LET hits = SELECT OSPath AS SourceFile, Size,
                        Mtime AS Modified, Ctime AS Changed,
                        Btime AS Created, Atime AS LastAccessed
        FROM glob(globs=specs.Glob, accessor=Accessor)
        WHERE NOT IsDir
         AND (Size <= MaxFileSize OR (log(message="Skipping...") AND FALSE))
      
      SELECT *, upload(file=SourceFile, accessor=Accessor, mtime=Modified) AS Upload
      FROM hits

Key glob patterns:

  • Windows: C:\Users\*\AppData\**\*.log or C:/Users/*/AppData/**/*.log
  • Linux: /home/*/.config/**/*
  • macOS: /Users/*/Library/Application Support/**/*
  • Use ** for recursive, * for single-level
  • Use {a,b,c} for alternatives: /path/*.{log,db,json}

Registry Collection (Windows)

-- Read a registry value
SELECT Data FROM reg_read(path='HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Run')

-- Glob registry keys
SELECT FullPath, Data, Mtime FROM glob(globs='HKEY_LOCAL_MACHINE\\SOFTWARE\\**', accessor='registry')

-- Parse specific values
SELECT Key, Data FROM reg_key(path='HKEY_LOCAL_MACHINE\\SOFTWARE\\MyApp')

SQLite Database Collection & Parsing

-- Standard pattern (from Windows.Applications.Chrome.History)
LET history_files = SELECT * FROM foreach(
  row={
     SELECT Uid, Name AS User, expand(path=Directory) AS HomeDirectory
     FROM Artifact.Windows.Sys.Users()
     WHERE Name =~ userRegex
  },
  query={
     SELECT User, OSPath, Mtime
     FROM glob(globs=historyGlobs, root=HomeDirectory)
  })

SELECT * FROM foreach(row=history_files,
  query={
    SELECT User, url AS visited_url, title, visit_count
    FROM sqlite(file=OSPath, query=urlSQLQuery)
  })

Key VQL function: sqlite(file=<path>, query=<SQL>) — queries a SQLite database on the endpoint without uploading it.

JSON Log Parsing

-- Parse JSON lines (one JSON object per line)
SELECT timestamp(epoch=atoi(string=timestamp_str)) AS Timestamp,
       message AS Message,
       level AS Level
FROM parse_json(filename=OSPath, accessor='file')

-- Or for newline-delimited JSON:
SELECT * FROM parse_lines(filename=OSPath)
WHERE line =~ '^{'
LET parsed <= parse_json(data=line)

-- For structured JSON files:
SELECT * FROM parse_json(filename=OSPath)

Process Listing with Filtering

-- Pattern from Linux.Network.NetstatEnriched
SELECT Pid, Name, Exe, CommandLine, Username
FROM pslist()
WHERE Name =~ ProcessNameRegex
  AND Exe =~ ProcessPathRegex
  AND CommandLine =~ CommandLineRegex
  AND Username =~ UsernameRegex

Network Connection Collection

-- Pattern from Linux.Network.NetstatEnriched
SELECT Laddr.IP AS Laddr, Laddr.Port AS Lport,
       Raddr.IP AS Raddr, Raddr.Port AS Rport,
       Pid, Status,
       process_tracker_get(id=Pid).Data AS ProcInfo,
       join(array=process_tracker_callchain(id=Pid).Data.Name, sep=" -> ") AS CallChain
FROM connections()
WHERE Status =~ ConnectionStatusRegex
 AND Raddr =~ IPRegex
 AND ProcInfo.Name =~ ProcessNameRegex

5. VQL Patterns Reference

Reading SQLite Databases

-- Basic query
SELECT * FROM sqlite(file='/path/to/db.sqlite', query='SELECT * FROM table_name')

-- With timestamp conversion (Chrome uses Windows filetime * 10)
SELECT timestamp(winfiletime=visit_time * 10) AS visit_time
FROM sqlite(file=OSPath, query=urlSQLQuery)

-- Parameterized SQL query
parameters:
  - name: urlSQLQuery
    default: |
      SELECT url, title, visit_count FROM urls

Parsing JSON Logs Line by Line

-- Single JSON object per file
SELECT * FROM parse_json(filename=OSPath)

-- NDJSON (one JSON per line)
SELECT * FROM parse_lines(filename=OSPath)
WHERE line =~ '^{'
-- Then parse each line:
LET Parsed <= parse_json(data=line)

Collecting File Metadata + Content

-- Metadata only (no upload)
SELECT OSPath, Size, Mtime, Ctime, Atime, Btime
FROM glob(globs=PathGlob)

-- Upload files with metadata
SELECT OSPath AS SourceFile, Size, Mtime AS Modified,
       upload(file=OSPath, accessor=Accessor) AS Upload
FROM glob(globs=PathGlob, accessor=Accessor)
WHERE NOT IsDir

-- Conditional upload with size limits
SELECT *, if(condition=Size < MaxFileSize,
        then=upload(file=OSPath),
        else=NULL) AS Upload
FROM glob(globs=PathGlob)
WHERE NOT IsDir

Cross-Platform Path Handling

-- Build paths for different OSes
LET WindowsPaths = SELECT * FROM glob(globs='C:\\Users\\*\\AppData\\Roaming\\**\\config.json')
LET LinuxPaths = SELECT * FROM glob(globs='/home/*/.config/**/config.json')
LET MacOSPaths = SELECT * FROM glob(globs='/Users/*/Library/Application Support/**/config.json')

-- Use expand() to resolve user home directories
SELECT expand(path=Directory) AS HomeDirectory FROM Artifact.Windows.Sys.Users()

-- Parse username from path (Windows)
parse_string_with_regex(regex="\\\\Users\\\\(?P<User>[^\\\\]+)", string=OSPath).User

-- Parse username from path (macOS)
parse_string_with_regex(regex="/Users/(?P<User>[^/]+)", string=OSPath).User

-- Use pathspec for NTFS raw access
LET RootPath <= pathspec(Path=Root, accessor='ntfs')

6. Structuring Large Multi-Source Artifacts

One Artifact vs Multiple

Use ONE artifact when:

  • All sources collect related data for the same forensic purpose (e.g., all AI tool evidence)
  • Sources share parameters and description
  • You want a single collection to gather everything
  • Results should appear in a unified report

Use MULTIPLE artifacts when:

  • Sources are independently useful (e.g., process listing vs browser history)
  • Different parameter sets are needed
  • Different users would collect them at different times
  • You want to minimize resource usage per collection

Multi-Source Architecture (Serial vs Parallel)

Serial mode (no source-level preconditions) — sources share scope:

sources:
  - name: Step1
    query: |
      LET X <= SELECT ...  -- X is visible to Step2
      SELECT * FROM ...
  - name: Step2
    query: |
      SELECT * FROM X     -- Can reference Step1's variables

Parallel mode (any source has precondition) — sources get independent scope:

sources:
  - name: WindowsData
    precondition: SELECT OS FROM info() WHERE OS = 'windows'
    query: |
      LET X <= ...  -- X is NOT visible to MacOSData
      SELECT * FROM ...
  - name: MacOSData
    precondition: SELECT OS FROM info() WHERE OS = 'darwin'
    query: |
      -- Cannot reference X from WindowsData
      SELECT * FROM ...

Using export for Shared VQL Across Sources

export: |
  LET CommonGlob <= glob(globs=PathGlob, accessor='auto')
  LET CommonFilter <= ... 

sources:
  - name: Source1
    query: |
      SELECT * FROM CommonFilter WHERE ...
  - name: Source2
    query: |
      SELECT * FROM CommonFilter WHERE ...

The export section's VQL is prepended to every source. Other artifacts can import it via:

imports:
  - Custom.AIToolEvidence

Recommended Pattern for AI Tool Forensic Artifact

name: Custom.AIToolEvidence
description: |
  Collects forensic evidence of AI tool usage across Windows, Linux, and macOS.
  
  Gathers configuration files, conversation logs, session data, and cache
  from AI coding assistants and chat tools.

type: CLIENT

export: |
  LET AIToolPaths <= SELECT * FROM glob(globs=SearchGlob, accessor='auto')
    WHERE NOT IsDir AND log(message="Found: " + OSPath)

parameters:
  - name: SearchGlob
    description: Glob pattern for AI tool files
    default: ...
  - name: UploadFiles
    type: bool
    default: Y
    description: Upload file content in addition to metadata

sources:
  - name: WindowsConfig
    precondition: SELECT OS FROM info() WHERE OS = 'windows'
    query: |
      ...

  - name: LinuxConfig
    precondition: SELECT OS FROM info() WHERE OS = 'linux'
    query: |
      ...

  - name: MacOSConfig
    precondition: SELECT OS FROM info() WHERE OS = 'darwin'
    query: |
      ...

  - name: ConversationLogs
    query: |
      SELECT OSPath, Size, Mtime, Modified,
             if(condition=UploadFiles,
                then=upload(file=OSPath)) AS Upload
      FROM AIToolPaths
      WHERE OSPath =~ 'conversation|history|chat|session'

column_types:
  - name: Mtime
    type: timestamp
  - name: Upload
    type: preview_upload

7. Artifact Parameters (Complete Reference)

Parameter Schema

parameters:
  - name: ParameterName          # Required - variable name in VQL
    friendly_name: Display Name  # Optional - shown in GUI
    description: Help text       # Optional - tooltip/description
    default: default_value       # Always a string, even for numbers
    type: string                 # Data type (see below)
    validating_regex: '^...$'   # Visual validation hint only
    choices:                     # For choices/multichoice types
      - OptionA
      - OptionB

All Parameter Types

Type VQL Conversion Description
string None (passthrough) Default type. Simple text input
int / integer / int64 LET x <= int(int=x) Integer input
float LET x <= parse_float(string=x) Float input
bool LET x <= x =~ '^(Y|TRUE|YES|OK)$' Checkbox
timestamp LET x <= timestamp(epoch=x) Date/time picker
regex None (string) Regex editor with suggestions
regex_array LET x <= parse_json_array(data=x) List of regex patterns
yara None (string) YARA rule editor with syntax highlighting
json LET x <= parse_json(data=x) JSON dict input
json_array LET x <= parse_json_array(data=x) JSON array input
csv LET x <= SELECT * FROM parse_csv(filename=x, accessor='data') Table-based CSV editor
choices None (string) Single-select dropdown
multichoice LET x <= parse_json_array(data=x) Multi-select checklist
hidden None (string) Not shown in GUI
redacted None Masked value display
upload LET x <= SELECT Content FROM http_client(url=x) File upload (<4MB)
upload_file Written to temp file File upload (any size, written to disk)
server_metadata LET x <= server_metadata().x Populated from server metadata
artifactset LET x <= SELECT * FROM parse_csv(...) Artifact name selector
xml LET x <= parse_xml(file=x, accessor="data") XML document input
yaml LET x <= parse_yaml(filename=x, accessor="data") YAML document input
starlark LET x <= starl(code=x) Starlark code block

Important: Default values are ALWAYS strings. Use "10" not 10 for int defaults.

Parameter Validation Pattern

parameters:
  - name: MaxFiles
    type: int
    default: "100"
    validating_regex: '^\d{1,5}$'

In VQL, add extra validation:

LET MaxFiles <= if(condition=NOT MaxFiles =~ '^\d{1,5}$',
                   then=100,
                   else=int(int=MaxFiles))

8. Report Generation

Reports Section

Artifacts can include a reports section that defines Go templates with embedded VQL for rendering results in the GUI.

reports:
  - type: CLIENT
    template: |
      {{ define "pre" }}
      LET Cap(X) = upcase(string=X[0:1]) + X[1:]
      {{ end }}
      {{ $_ := Query "pre" | Expand }}

      # Results for {{ Scope "Hostname" }}
      {{ Query "SELECT * FROM source(source='BasicInformation')" | Table }}

Report Template Features

  • Go template syntax with VQL integration
  • {{ Query "..." | Table }} — render query results as table
  • {{ Query "..." | TimeChart "column" }} — render time chart
  • {{ Scope "varname" }} — access scope variables
  • {{ if condition }} ... {{ end }} — conditional rendering
  • {{ define "name" }} ... {{ end }} — define reusable blocks
  • {{ range Query "..." | Expand }} ... {{ end }} — iterate over results

Column Types

Control how columns are displayed in the GUI:

column_types:
  - name: Timestamp
    type: timestamp
  - name: Upload
    type: preview_upload
  - name: ChildrenTree
    type: tree

Reference: Real Artifact Examples Studied

1. Windows.Applications.Chrome.History — SQLite + glob + user enumeration

  • Uses Artifact.Windows.Sys.Users() to find user directories
  • Globs for History files per user
  • Queries SQLite with parameterized SQL
  • Shows pattern: glob → foreach → sqlite()

2. MacOS.Applications.Chrome.History — macOS variant

  • Uses parse_string_with_regex() to extract username from path
  • Simpler glob pattern (no user enumeration needed)
  • Direct glob(globs=historyGlobs) + foreach → sqlite()

3. Generic.Client.Info — Multi-OS with source-level preconditions + reports

  • 5 sources with different preconditions (BasicInformation, LinuxInfo, WindowsInfo, Users)
  • Includes reports: section with Go templates
  • Shows column_types configuration
  • Uses export for shared VQL

4. Generic.Collectors.File — CSV-parameterized file collection

  • Uses csv type parameter for flexible glob patterns
  • Multi-source (metadata + uploads) in serial mode
  • Shows pathspec(), upload(), and workers= for parallel uploads

5. Generic.Detection.Yara.Glob — Cross-platform with aliases

  • Uses aliases field for OS-specific names
  • upload type parameter for YARA rules
  • Conditional upload with if(condition=UploadHits, then=upload_hits, else=hits)
  • Shows column_types: preview_upload

6. Linux.Network.NetstatEnriched — Filtering-heavy artifact

  • 8 regex parameters for flexible filtering
  • Uses connections() + process_tracker_get() + process_tracker_callchain()
  • column_types: tree for process hierarchy

Key VQL Plugins/Functions Reference

Purpose VQL Function/Plugin
File globbing glob(globs=..., accessor=...)
SQLite query sqlite(file=..., query=...)
JSON parsing parse_json(filename=...) or parse_json(data=...)
Line-by-line parsing parse_lines(filename=...)
File upload upload(file=..., accessor=...)
Registry read reg_read(path=...), reg_key(path=...)
Process list pslist()
Network connections connections()
System info info()
User enumeration Artifact.Windows.Sys.Users()
Path building pathspec(Path=..., accessor=...)
Path parsing parse_string_with_regex(regex=..., string=...)
Conditional logic if(condition=..., then=..., else=...)
Logging log(message=..., args=[...])
Type checking typeof(a=variable)
Process tracker process_tracker_get(id=...), process_tracker_callchain(id=...)
Timestamps timestamp(epoch=...), timestamp(winfiletime=...)

Accessor Types (for glob(), upload(), etc.)

Accessor Purpose
auto Auto-selects appropriate accessor for OS
file Regular filesystem
ntfs Raw NTFS (bypasses file locks, can read alternate data streams)
registry Windows registry
data In-memory data (for parsing CSV/JSON parameters)
scope Access query results as files

Summary of Key Design Decisions for AI Tool Evidence Artifact

  1. Use type: CLIENT — this runs on endpoints
  2. Use source-level preconditions for OS branching — gives clean per-OS result tables
  3. Use export: section for shared VQL (common glob patterns, parsing functions)
  4. Use parameters: with sensible defaults — regex type for filtering, csv type for file paths, bool for upload toggle
  5. Structure sources by data category rather than by OS if results should be combined; by OS if each OS has very different data
  6. Use column_types to ensure timestamps and uploads render correctly in GUI
  7. Include precondition even on top-level if artifact is OS-specific
  8. Follow naming convention: Custom.AITools.{ToolName}.{DataType}
  9. Add aliases if creating per-OS variants that should appear under multiple search paths
  10. Test with velociraptor gui on each target OS before deploying