Command Line Parameters for logagent
Synopsis
logagent [options] [file list]
Options | Description |
---|---|
Genernal options | |
-h, --help | output logagent help |
-V, --version | output logagent version |
-v, --verbose | output activity report every minute |
--geoip |
true/false to enable/disable geoip lookups in patterns. |
--logsene-tmp-dir path | directory to store status and buffered logs (during network outage) |
-f, --file |
file with pattern definitions |
-s, --suppress | silent, print no logs to stdout, prints only stats on exit |
--print_stats | print processing stats in the given interval in seconds, e.g. --print_stats 30 to stderr. Usefull with -s to see logagent activity on the console without printing the parsed logs to stdout. |
Log input options | |
-g glob-pattern | use a glob pattern to watch log files e.g. -g "{/var/log/*.log,/Users/stefan/myapp/*.log}" . The complete glob expression must be quoted, to avoid interpretation of special characters by the linux shell. |
--stdin | read from stdin, default if no other input like files or UDP are set |
-n name | name for the log source only when stdin is used, important to make multi-line patterns working on stdin because the status is tracked by the log source name. |
-u UDP_PORT | starts a syslogd UDP listener on the given port to act as syslogd |
--heroku PORT | listens for Heroku logs (http drain / framed syslog over http) |
--cfhttp PORT | listens for Cloud Foundry logs (syslog over http) |
list of files | Every argument after the options list is interpreted as file name. All files in the file list (e.g. /var/log/*.log) are watched by tail-forever starting at end of file |
Output options | |
standard output stream | combine logagent with any unix tool via pipes |
-y, --yaml | prints parsed messages in YAML format to stdout |
-p, --pretty | prints parsed messages in pretty json format to stdout |
-j, --ldjson | print parsed messages in line delimited JSON format to stdout |
Elasticsearch / Logsene | Log storage |
-e, --elasticsearch-host |
Elasticsearch url e.g. http://localhost:9200, default htpps://logsene-receiver.sematext.com:443' |
-t, --index |
Logsene App Token to insert parsed records into Logsene or Elasticsearch index (see --elasticsearch-host) |
--http-proxy |
HTTP proxy url |
--https-proxy |
HTTPS proxy url |
rtail | Realtime log viewer |
--rtail-port | forwards logs via UDP to rtail server |
--rtail-host hostname | rtail server (UI for realtime logs), default: localhost |
--rtail-web-port |
starts rtail UI webserver (if not installed install with: - npm i rtail -g) |
--rtail-web-host |
rtail UI webserver and bind hostname. E.g. logagent --rtail-web-port 9000 --rtail-port 8989 --rtail-web-host $(hostname) -g \'/var/log/**/*.log |
The default output is line delimited JSON for parsed log lines, as long as no format options like -yaml (YAML format), -p (pretty JSON), or -s (silent, no output to console) are specified.
Environment variables
Variable | Description |
---|---|
LOGSENE_TMP_DIR | Directory to store failed bulk requests, for later re-transmission. |
LOGSENE_LOG_INTERVAL | Time to batch logs before a bulk request is done. Default 10000 ms (10 seconds) |
LOGSENE_BULK_SIZE | Maximum size of a bulk request. Default 1000. |
LOGSENE_URL | URL for the Logsene receiver. For a local Elasticsearch server or for On-Premise version of Logsene. Defaults to Sematext Logsene SaaS receiver https://logsene-receiver.sematext.com/_bulk. Example for Elasticsearch: LOGSENE_URL=http://localhost:9200/_bulk |
HTTPS_PROXY | Proxy URL for HTTPS endpoints, like Logsene receiver. export HTTPS_PROXY=http://my-proxy.example |
HTTP_PROXY | Proxy URL for HTTP endpoints (e.g. On-Premises or local Elasticsearch). export HTTP_PROXY=http://my-proxy.example |
LOGAGENT_CONFIG | Filename to read logagent CLI parameters from a file, defaults to `/etc/sematext/logagent.conf |
PATTERN_MATCHING_ENABLED | Default is 'true'. The value 'false' disables parsing of logs. |
Command Line Examples
# Be Evil: parse all logs
# stream logs to Logsene 1-Click ELK stack
logagent -t LOGSENE_TOKEN /var/log/*.log
# Act as syslog server on UDP and forward messages to Logsene
logagent -u 514 -t LOGSENE_TOKEN
# Act as syslog server on UDP and write YAML formatted messages to console
logagent -u 514 -y
Use a glob pattern to build the file list
logagent -t LOGSENE_TOKEN -g '/var/log/**/*.log'
# pass multiple glob patterns
logagent -t LOGSENE_TOKEN -g '{/var/log/*.log,/opt/myapp/*.log}'
Watch selective log output on console by passing logs via stdin and format in YAML
tail -f /var/log/access.log | logagent -y -n httpd
tail -f /var/log/system.log | logagent -f my_own_patterns.yml -y
Ship logs to rtail and Logsene to view logs in real-time in rtail and store logs in Logsene
# rtail don't need to be installed, logagent uses the rtail protocol
logagent -t $LOGSENE_TOKEN --rtail-host myrtailserver --rtail-port 9999 /var/log/*.log
Logagent can start the rtail web-server (in-process, saving memory), open browser with http://localhost:8080
# logagent has no dependency to rtail, to keep the package small
sudo npm i rtail -g
logagent -s -t $LOGSENE_TOKEN --rtail-web-port 8080 --rtail-port 9999 /var/log/*.log
And of course you can combine rtail and Logagent in the traditional way, simply connect both via unix pipes. An example with rtail and Logsene storage and charts: