Slack Chat • Forums • Mailing List
Vector is a [high-performance][docs.performance] observability data router. It makes [collecting][docs.sources], [transforming][docs.transforms], and [sending][docs.sinks] logs, metrics, and events easy. It decouples data collection & routing from your services, giving you control and data ownership, among [many other benefits][docs.use_cases].
Built in [Rust][url.rust], Vector places high-value on [performance][docs.performance], [correctness][docs.correctness], and [operator friendliness][docs.administration]. It compiles to a single static binary and is designed to be [deployed][docs.deployment] across your entire infrastructure, serving both as a light-weight [agent][docs.agent_role] and a highly efficient [service][docs.service_role], making the process of getting data from A to B simple and unified.
- [Use cases][docs.use_cases]
- [Concepts][docs.concepts]
- [Data model][docs.data_model]
- [Guarantees][docs.guarantees]
- [Installation][docs.installation] - [docker][docs.docker], [apt][docs.apt], [homebrew][docs.homebrew], [yum][docs.yum], and [more][docs.installation]
- [Getting started][docs.getting_started]
- [Deployment][docs.deployment] - [topologies][docs.topologies], [roles][docs.roles]
- [Configuration][docs.configuration] - [sources][docs.sources], [transforms][docs.transforms], [sinks][docs.sinks]
- [Administration][docs.administration] - [starting][docs.starting], [stopping][docs.stopping], [reloading][docs.reloading], [updating][docs.updating]
- [Guides][docs.guides]
- [Community][url.community] - [forum][url.vector_forum], [slack chat][url.vector_chat], [mailing list][url.mailing_list]
- [Roadmap][url.roadmap] - [vote on new features][url.vote_feature]
- Fast - Built in [Rust][url.rust], Vector is [fast and memory efficient][docs.performance]. No runtime. No garbage collector.
- Correct - Obsessed with [getting the details right][docs.correctness].
- Vendor Neutral - Does not favor a specific storage. Fair, open, with the user's best interest in mind.
- Agent Or Service - One simple tool to get data from A to B. Deploys as an [agent][docs.agent_role] or [service][docs.service_role].
- Logs, Metrics, or Events - Logs, metrics, and events. Collect, unify, and ship all observability data.
- Clear Guarantees - A [guarantee support matrix][docs.guarantees] helps you understand your tradeoffs.
- Easy To Deploy - Cross-compiles to a single static binary with no runtime.
- Hot Reload - [Reload configuration on the fly][docs.reloading], without skipping a beat.
| Test | Vector | Filebeat | FluentBit | FluentD | Logstash | SplunkUF | SplunkHF |
|---|---|---|---|---|---|---|---|
| TCP to Blackhole | 86mib/s | n/a | 64.4mib/s | 27.7mib/s | 40.6mib/s | n/a | n/a |
| File to TCP | 76.7mib/s | 7.8mib/s | 35mib/s | 26.1mib/s | 3.1mib/s | 40.1mib/s | 39mib/s |
| Regex Parsing | 13.2mib/s | n/a | 20.5mib/s | 2.6mib/s | 4.6mib/s | n/a | 7.8mib/s |
| TCP to HTTP | 26.7mib/s | n/a | 19.6mib/s | <1mib/s | 2.7mib/s | n/a | n/a |
| TCP to TCP | 69.9mib/s | 5mib/s | 67.1mib/s | 3.9mib/s | 10mib/s | 70.4mib/s | 7.6mib/s |
To learn more about our performance tests, please see the [Vector test harness][url.test_harness].
| Test | Vector | Filebeat | FluentBit | FluentD | Logstash | Splunk UF | Splunk HF |
|---|---|---|---|---|---|---|---|
| Disk Buffer Persistence | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | |
| File Rotate (create) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| File Rotate (copytruncate) | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ |
| File Truncation | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Process (SIGHUP) | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | |
| JSON (wrapped) | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ |
To learn more about our performance tests, please see the [Vector test harness][url.test_harness].
Run the following in your terminal, then follow the on-screen instructions.
curl https://sh.vector.dev -sSf | shOr view [platform specific installation instructions][docs.installation].
| Name | Description |
|---|---|
[file][docs.file_source] |
Ingests data through one or more local files and outputs [log][docs.log_event] events. |
[statsd][docs.statsd_source] |
Ingests data through the StatsD UDP protocol and outputs [log][docs.log_event] events. |
[stdin][docs.stdin_source] |
Ingests data through standard input (STDIN) and outputs [log][docs.log_event] events. |
[syslog][docs.syslog_source] |
Ingests data through the Syslog 5424 protocol and outputs [log][docs.log_event] events. |
[tcp][docs.tcp_source] |
Ingests data through the TCP protocol and outputs [log][docs.log_event] events. |
[vector][docs.vector_source] |
Ingests data through another upstream Vector instance and outputs [log][docs.log_event] events. |
[+ request a new source][url.new_source]
| Name | Description |
|---|---|
[add_fields][docs.add_fields_transform] |
Accepts [log][docs.log_event] events and allows you to add one or more fields. |
[field_filter][docs.field_filter_transform] |
Accepts [log][docs.log_event] and [metric][docs.metric_event] events and allows you to filter events by a field's value. |
[grok_parser][docs.grok_parser_transform] |
Accepts [log][docs.log_event] events and allows you to parse a field value with [Grok][url.grok]. |
[json_parser][docs.json_parser_transform] |
Accepts [log][docs.log_event] events and allows you to parse a field value as JSON. |
[lua][docs.lua_transform] |
Accepts [log][docs.log_event] events and allows you to transform events with a full embedded [Lua][url.lua] engine. |
[regex_parser][docs.regex_parser_transform] |
Accepts [log][docs.log_event] events and allows you to parse a field's value with a [Regular Expression][url.regex]. |
[remove_fields][docs.remove_fields_transform] |
Accepts [log][docs.log_event] and [metric][docs.metric_event] events and allows you to remove one or more event fields. |
[sampler][docs.sampler_transform] |
Accepts [log][docs.log_event] events and allows you to sample events with a configurable rate. |
[tokenizer][docs.tokenizer_transform] |
Accepts [log][docs.log_event] events and allows you to tokenize a field's value by splitting on white space, ignoring special wrapping characters, and zipping the tokens into ordered field names. |
[+ request a new transform][url.new_transform]
| Name | Description |
|---|---|
[aws_cloudwatch_logs][docs.aws_cloudwatch_logs_sink] |
Batches [log][docs.log_event] events to [AWS CloudWatch Logs][url.aws_cw_logs] via the PutLogEvents API endpoint. |
[aws_kinesis_streams][docs.aws_kinesis_streams_sink] |
Batches [log][docs.log_event] events to [AWS Kinesis Data Stream][url.aws_kinesis_data_streams] via the PutRecords API endpoint. |
[aws_s3][docs.aws_s3_sink] |
Batches [log][docs.log_event] events to [AWS S3][url.aws_s3] via the PutObject API endpoint. |
[blackhole][docs.blackhole_sink] |
Streams [log][docs.log_event] and [metric][docs.metric_event] events to a blackhole that simply discards data, designed for testing and benchmarking purposes. |
[console][docs.console_sink] |
Streams [log][docs.log_event] and [metric][docs.metric_event] events to the console, STDOUT or STDERR. |
[elasticsearch][docs.elasticsearch_sink] |
Batches [log][docs.log_event] events to [Elasticsearch][url.elasticsearch] via the _bulk API endpoint. |
[http][docs.http_sink] |
Batches [log][docs.log_event] events to a generic HTTP endpoint. |
[kafka][docs.kafka_sink] |
Streams [log][docs.log_event] events to [Apache Kafka][url.kafka] via the [Kafka protocol][url.kafka_protocol]. |
[prometheus][docs.prometheus_sink] |
Pulls [metric][docs.metric_event] events to [Prometheus][url.prometheus] metrics service. |
[splunk_hec][docs.splunk_hec_sink] |
Batches [log][docs.log_event] events to a [Splunk HTTP Event Collector][url.splunk_hec]. |
[tcp][docs.tcp_sink] |
Streams [log][docs.log_event] events to a TCP connection. |
[vector][docs.vector_sink] |
Streams [log][docs.log_event] events to another downstream Vector instance. |
[+ request a new sink][url.new_sink]
Copyright 2019, Vector Authors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use these files except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Developed with ❤️ by Timber.io