Camel Components

Grok

Since Camel 3.0

This component provides dataformat for processing inputs with grok patterns. Grok patterns are used to process unstructured data into structured objects: a Map<String, Object> of the captures for one match, a List of such Maps for several matches, or no body when nothing matches. With a split, as in the examples below, an input with one match gives a Map whose entries are split one by one.

This component is based on the Java Grok library

Maven users will need to add the following dependency to their pom.xml for this component:

<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-grok</artifactId>
    <version>x.x.x</version>
    <!-- use the same version as your Camel core version -->
</dependency>

Usage

Extract all IP addresses from input

Java
XML
YAML

from("direct:in")
    .unmarshal().grok("%{IP:ip}")
    .to("log:out");

<route>
  <from uri="direct:in"/>
  <unmarshal><grok pattern="%{IP:ip}"/></unmarshal>
  <to uri="log:out"/>
</route>

- route:
    from:
      uri: direct:in
      steps:
        - unmarshal:
            grok:
              pattern: "%{IP:ip}"
        - to:
            uri: log:out

Parse Apache logs and process only 4xx responses

Java
XML
YAML

from("file://apacheLogs")
    .unmarshal().grok("%{COMBINEDAPACHELOG}")
    .split(body()).filter(simple("${body[response]} starts with '4'"))
    .to("log:4xx");

<route>
  <from uri="file://apacheLogs"/>
  <unmarshal><grok pattern="%{COMBINEDAPACHELOG}"/></unmarshal>
  <split>
    <simple>${body}</simple>
    <filter>
      <simple>${body[response]} starts with '4'</simple>
      <to uri="log:4xx"/>
    </filter>
  </split>
</route>

- route:
    from:
      uri: file://apacheLogs
      steps:
        - unmarshal:
            grok:
              pattern: "%{COMBINEDAPACHELOG}"
        - split:
            simple: "${body}"
            steps:
              - filter:
                  simple: "${body[response]} starts with '4'"
                  steps:
                    - to:
                        uri: log:4xx

Preregistered patterns

This component comes with preregistered patterns, which are based on Logstash patterns. All Java Grok Default Patterns are preregistered and as such could be used without manual registration.

Custom patterns

Camel Grok DataFormat supports plugable patterns, which are auto loaded from Camel Registry. You can register patterns with Java DSL and Spring DSL:

Java-only: registering a custom Grok pattern

bindToRegistry("myCustomPatternBean", new GrokPattern("FOOBAR", "foo|bar"));

from("direct:in")
    .unmarshal().grok("%{FOOBAR:fooBar}")
    .to("log:out");

In Spring XML, you register the custom pattern as a bean:

XML-only: Spring XML bean registration

<beans>
    <bean id="myCustomPatternBean" class="org.apache.camel.component.grok.GrokPattern">
        <constructor-arg value="FOOBAR"/>
        <constructor-arg value="foo|bar"/>
    </bean>
</beans>

Grok Data format Options

The Grok dataformat supports the following options which are listed below.

Name Default Java Type Description

Name	Default	Java Type	Description
pattern (common)		`String`	Required The grok pattern to match lines of input.
flattened (common)	`false`	`Boolean`	Whether to use flattened mode. In flattened mode an exception is thrown when there are multiple pattern matches with the same key.
allowMultipleMatchesPerLine (common)	`true`	`Boolean`	Whether to allow multiple matches per line. If false, every line of input is matched for the pattern only once. Otherwise the line can be scanned multiple times when a non-terminal pattern is used.
namedOnly (common)	`false`	`Boolean`	Whether to capture named expressions only or not (i.e. %\{IP:ip} but not ${IP}).

pattern (common)

String

Required The grok pattern to match lines of input.

flattened (common)

false

Boolean

Whether to use flattened mode. In flattened mode an exception is thrown when there are multiple pattern matches with the same key.

allowMultipleMatchesPerLine (common)

true

Boolean

Whether to allow multiple matches per line. If false, every line of input is matched for the pattern only once. Otherwise the line can be scanned multiple times when a non-terminal pattern is used.

namedOnly (common)

false

Boolean

Whether to capture named expressions only or not (i.e. %\{IP:ip} but not ${IP}).

Message Body

The message body at each side of the Grok data format: Marshal input and Unmarshal input are the bodies marshal and unmarshal can take, and Marshal output and Unmarshal output the body after them.

Side Body Description

Side	Body	Description
Unmarshal input	`text`, `bytes`, `stream` or `file`, as text	Lines of text, such as a log file: each line is matched with the pattern.
Unmarshal output	`map`, `list` or `null`	A Map of the captures for one match, a List of such Maps for several, or null when no line matches. The Maps cannot be modified.

Unmarshal input

text, bytes, stream or file, as text

Lines of text, such as a log file: each line is matched with the pattern.

Unmarshal output

map, list or null

A Map of the captures for one match, a List of such Maps for several, or null when no line matches. The Maps cannot be modified.

The bodies are the ones with the default options. A side that is not listed is not known, and tooling does not guess it. The carriers are described in the Camel Catalog.