121 lines
4.6 KiB
Text
121 lines
4.6 KiB
Text
[role="xpack"]
|
|
[[grokdebugger-getting-started]]
|
|
|
|
ifndef::gs-mini[]
|
|
=== Getting Started with the Grok Debugger
|
|
endif::gs-mini[]
|
|
|
|
ifdef::gs-mini[]
|
|
== Getting Started with the Grok Debugger
|
|
endif::gs-mini[]
|
|
|
|
++++
|
|
<titleabbrev>Getting Started</titleabbrev>
|
|
++++
|
|
|
|
TIP: See the documentation about the ingest node
|
|
{ref}/grok-processor.html[grok processor] and the Logstash {logstash-ref}/plugins-filters-grok.html[grok filter] more info about grok.
|
|
|
|
The Grok Debugger is automatically enabled in {kib}. It is located under the *DevTools* tab in {kib}.
|
|
|
|
To start debugging grok patterns:
|
|
|
|
. Open Kibana in your web browser and log in. If you are running Kibana
|
|
locally, go to `http://localhost:5601/`.
|
|
|
|
. Click **DevTools** in the side navigation and then click **Grok Debugger**
|
|
on the top navigation bar.
|
|
+
|
|
image::dev-tools/grokdebugger/images/grok-debugger.png["Grok Debugger UI"]
|
|
|
|
. Under Sample Data, enter a sample message that is representative of the data you
|
|
want to parse. For example:
|
|
+
|
|
[source,ruby]
|
|
-------------------------------------------------------------------------------
|
|
55.3.244.1 GET /index.html 15824 0.043
|
|
-------------------------------------------------------------------------------
|
|
|
|
. Under Grok Pattern, enter the grok pattern that you want to apply to the data.
|
|
+
|
|
For example, to parse the log line in the example, you use:
|
|
+
|
|
[source,ruby]
|
|
-------------------------------------------------------------------------------
|
|
%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}
|
|
-------------------------------------------------------------------------------
|
|
|
|
. Click the **Simulate** button.
|
|
+
|
|
Under Structured Data, you'll see the simulated event that results from applying the grok
|
|
pattern:
|
|
+
|
|
image::dev-tools/grokdebugger/images/grok-debugger-output.png["Viewing Grok Debugger Output"]
|
|
+
|
|
Any errors in the pattern will appear at the top of the page. For example,
|
|
here you see a parse exception because the pattern name is misspelled as `WORDD`
|
|
and therefore can't be found in the pattern dictionary:
|
|
+
|
|
image::dev-tools/grokdebugger/images/grok-debugger-error.png["Viewing Grok Debugger Errors"]
|
|
+
|
|
You can click the **More** link to see more detail about the message.
|
|
+
|
|
Click **OK** to dismiss the message and continue iterating over the grok pattern
|
|
until there are no errors and the output matches the event that you expect.
|
|
|
|
//TODO: Update LS and ingest node docs with pointers to the new grok debugger. Replace references to the Heroku app.
|
|
|
|
[float]
|
|
[[grokdebugger-custom-patterns]]
|
|
==== Testing Custom Patterns
|
|
|
|
If the default grok pattern dictionary doesn't contain the patterns you need,
|
|
you may need to define custom patterns. You can use the Grok Debugger to test
|
|
and debug customer patterns.
|
|
|
|
The custom patterns that you enter in the Grok Debugger are not saved. They're
|
|
only available for the current debugging session and have no side effects.
|
|
|
|
To test a custom pattern:
|
|
|
|
. Repeat the steps that you followed previously to enter the sample message and
|
|
grok pattern. For this example, let's use the following sample message:
|
|
+
|
|
[source,ruby]
|
|
-------------------------------------------------------------------------------
|
|
Jan 1 06:25:43 mailserver14 postfix/cleanup[21403]: BEF25A72965: message-id=<20130101142543.5828399CCAF@mailserver14.example.com>
|
|
-------------------------------------------------------------------------------
|
|
+
|
|
And this grok pattern:
|
|
+
|
|
[source,ruby]
|
|
-------------------------------------------------------------------------------
|
|
%{SYSLOGBASE} %{POSTFIX_QUEUEID:queue_id}: %{MSG:syslog_message}
|
|
-------------------------------------------------------------------------------
|
|
+
|
|
Notice that the grok pattern references custom patterns called `POSTFIX_QUEUEID`
|
|
and `MSG`.
|
|
|
|
. Expand **Custom Patterns** and enter pattern definitions for any custom
|
|
patterns that you want to use in the grok expression. Each pattern definition
|
|
must be specified on its own line.
|
|
+
|
|
For the grok pattern in the example, you need to specify pattern definitions
|
|
for `POSTFIX_QUEUEID` and `MSG`:
|
|
+
|
|
[source,ruby]
|
|
-------------------------------------------------------------------------------
|
|
POSTFIX_QUEUEID [0-9A-F]{10,11}
|
|
MSG message-id=<%{GREEDYDATA}>
|
|
-------------------------------------------------------------------------------
|
|
|
|
. Click the **Simulate** button.
|
|
+
|
|
Under Output, you'll see the simulated output event that results from applying
|
|
the grok pattern that contains the custom pattern:
|
|
+
|
|
image::dev-tools/grokdebugger/images/grok-debugger-custom-pattern.png["Debugging a custom pattern"]
|
|
+
|
|
If an error occurs, you can view the error message and continue iterating over
|
|
the custom pattern until there are no errors and the output matches the event
|
|
that you expect.
|