Welcome to the Splunk Add-on for JIRA Atlassian Service Desk application documentation¶

The Splunk Add-on for JIRA Atlassian Service Desk provides alerts action for JIRA issues creation:

Trigger JIRA issue creation from Splunk core alerts and Enterprise Security correlation searches
Dynamic retrieval per JIRA project for types of issues and priority
Dynamic assignment of priority (optional)
Dynamic and/or static assignment of summary, description, assignee and labels
Custom fields full capabilities via the embedded custom field structure in alerts (optional)
Deduplication feature workflow with bi-directional integration, allows detecting a duplication issue creation request, and adding new comments automatically instead of creating duplicated issues
Attaching Splunk alert results to the JIRA issue in CSV or JSON format
Resilient store JIRA issue creation, shall a JIRA issue fails to be created, the resilient workflow handles automatic retries with a resilient policy
Monitoring of JIRA issue workflow via the embedded Overview dashboard and out of the box alerts
Get any information from JIRA via the REST API custom command wrapper, generate and index to summary events or the metric store issues statistics per projects

Overview:¶

About¶

Author: Guilhem Marchand, Splunk certified consultant and part of Splunk Professional Services
First public release published in April 2020
License: Apache License 2.0

Compatibility¶

Splunk compatibility¶

This application is compatible with Splunk 8.0.x and later.

The latest release for Splunk 7.x is available in Git: - https://github.com/guilhemmarchand/TA-jira-service-desk-simple-addon/releases/tag/1.0.30

Splunk Enterprise Security compatibility¶

This application has been verified with ES 5.x and 6.x.

Python support¶

From the version 2.0.0, only Python3 is supported. (for Python2 support, use the latest release 1.0.x)

Web Browser compatibility¶

The application can be used with any of the supported Web Browser by Splunk:

https://docs.splunk.com/Documentation/Splunk/latest/Installation/Systemrequirements

JIRA compatibility¶

The Add-on is compatible with:

JIRA Server
JIRA Cloud
JIRA Data center

Support & donate¶

I am supporting my applications for free, for the good of everyone and on my own private time. As you can guess, this is a huge amount of time and efforts.

If you enjoy it, and want to support and encourage me, buy me a coffee (or a Pizza) and you will make me very happy!

This application is community supported.

Splunk Community¶

Open a question in Splunk answers for the application:

https://community.splunk.com

Splunk community slack¶

Contact me on Splunk community slack, and even better, ask the community!

https://splunk-usergroups.slack.com

Open a issue in Git¶

To report an issue, request a feature change or improvement, please open an issue in Github:

https://github.com/guilhemmarchand/TA-jira-service-desk-simple-addon/issues

Email support¶

guilhem.marchand@gmail.com

However, previous options are far betters, and will give you all the chances to get a quick support from the community of fellow Splunkers.

Download¶

The Splunk application can be downloaded from:

Splunk base¶

https://splunkbase.splunk.com/app/4958

GitHub¶

https://github.com/guilhemmarchand/TA-jira-service-desk-simple-addon

Deployment and configuration:¶

Deployment & Upgrades¶

Deployment matrix¶

Splunk roles	required
Search head	yes
Indexer tiers	no

If Splunk search heads are running in Search Head Cluster (SHC), the Splunk application must be deployed by the SHC deployer.

Dependencies¶

There are currently no dependencies for the application.

However, if you deploy the Splunk_SA_CIM package, make sure you have declared the cim_modactions index as the Add-on logs would automatically be directed to this index is the SA CIM application is installed on the search heads.

If the Splunk_SA_CIM is not installed, the Add-on logs will be generated in the _internal index. (This is a normal behaviour for Add-on developped with the Splunk Add-on builder that provide adaptive response capabilities)

Initial deployment¶

The deployment of the Splunk application is very straight forward:

Using the application manager in Splunk Web (Settings / Manages apps)
Extracting the content of the tgz archive in the “apps” directory of Splunk
For SHC configurations (Search Head Cluster), extract the tgz content in the SHC deployer and publish the SHC bundle

Upgrades¶

Upgrading the Splunk application is pretty much the same operation than the initial deployment.

All of TrackMe components and configuration items are upgraded resilient, in respects with Splunk configuration good practices.

Upgrade from version 1.x.x to 2.x.x¶

Warning

BREAKING CHANGES!

The major release 2.0 migrates from the Splunk Add-on Builder framework to the Splunk add-on-ucc-framework.
This fundamentally changes the way accounts are handled automatically, which means that once the upgrade has been performed you need to re-create your account(s) defining the connectivity to JIRA before alert actions can trigger again.

Proceed as follows:

Upgrade the Add-on to the latest release 2.x available
Restart the Splunk search head (or automatic rolling restart in Search Head Cluster)
Access to the configuration page, and re-create your connection to JIRA (not that in version 2.x you can setup multiple accounts)
Verify that the connection is successful
Optionnally verify either that an existing alert can trigger a JIRA ticket, or create a temporary test alert

Configuration¶

Configuring in Splunk Web¶

In normal circumstances, you will perform the configuration of the Add-on via the configuration UI in Splunk Web.

You cannot perform the account configuration manually by creating the expecting configuration files, these configuration items need to be handled and encrypted automatically.

In a Search Head Cluster context, the generated configuration is automatically replicated across the members of the cluster.

Configuration page:

JIRA account configuration¶

The Add-on for JIRA supports multiple accounts to be configured, an account consists in the following parameters:

account name: a name of your choice to represent this instance of JIRA
JIRA URL: the URL value which will be used for this instance, in the format <address>:<port>
Authentication type: currently the Add-on supports basic authenticaton (user/password) and PAT (Personal Access Token)
SSL related configuration items: SSL certificate verification and SSL certificate bundle

Configuring an account with Personal Access Token (PAT)¶

The recommended way for the authentication to JIRA with the Add-on is to use Personal Access Tokens:

https://confluence.atlassian.com/enterprise/using-personal-access-tokens-1026032365.html

Example in JIRA Software:

Add-on PAT account configuration:

Enter the PAT ID
Enter the PAT (stored encrypted)
Select “Personal Access Token” in authentication method

Configuring an account with basic authentication¶

Basic authentication relies on a traditional combo of username / password credentials:

Add-on PAT account configuration:

Enter the username
Enter the password (stored encrypted)
Select “Basic” in authentication method

Configuring via REST API¶

Alternatively, and this can be useful if for some reason you cannot access to the configuration UI (no end dead loop), the configuration can easily be achieved via REST calls to the Splunk API with curl.

Configuring the JIRA instance via curl¶

PAT authentication (Personal Access Token)

Assuming:

Account name: LAB
JIRA instance URL: myjira.mydomain.com:8443
Authentication method: pat
JIRA PAT ID: splunk_svc
JIRA PAT: xxxxxxxxxxx
SSL certificate verification: disabled (0=disabled, 1=enabled)

You would run the following curl command, either locally on a search head (in SHC, this will be replicated automatically), or remotely reaching out to a search head:

curl -k -u admin:'ch@ngeM3' -X POST https://localhost:8089/servicesNS/nobody/TA-jira-service-desk-simple-addon/ta_service_desk_simple_addon_account -d 'name=LAB' -d 'jira_url=myjira.mydomain.com:8443' -d 'auth_type=basic' -d 'username=splunk_svc' -d 'password=xxxxxxxxxxx' -d 'jira_auth_mode=pat' -d 'jira_ssl_certificate_validation=0'

basic authentication (username / password)

Assuming:

Account name: LAB
JIRA instance URL: myjira.mydomain.com:8443
Authentication type: basic
JIRA login username: admin
JIRA password: ch@ngeM3
SSL certificate verification: disabled (0=disabled, 1=enabled)

You would run the following curl command, either locally on a search head (in SHC, this will be replicated automatically), or remotely reaching out to a search head:

curl -k -u admin:'ch@ngeM3' -X POST https://localhost:8089/servicesNS/nobody/TA-jira-service-desk-simple-addon/ta_service_desk_simple_addon_account -d 'name=LAB' -d 'jira_url=myjira.mydomain.com:8443' -d 'auth_type=basic' -d 'username=admin' -d 'password=ch@ngeM3' -d 'jira_auth_mode=basic' -d 'jira_ssl_certificate_validation=0'

List account configured:

You can review your settings with a GET: (password and tokens are encrypted and not visible)

curl -k -u admin:'ch@ngeM3' -X GET https://localhost:8089/servicesNS/nobody/TA-jira-service-desk-simple-addon/ta_service_desk_simple_addon_account

Enabling the passthrough mode¶

To enable the passthrough mode:

curl -k -u admin:'ch@ngeM3' -X POST https://localhost:8089/servicesNS/nobody/TA-jira-service-desk-simple-addon/ta_service_desk_simple_addon_settings/advanced_configuration -d 'jira_passthrough_mode=1'

Setting the KVstore instance and port¶

When running in client mode, set the KVstore remote instance:

curl -k -u admin:'ch@ngeM3' -X POST https://localhost:8089/servicesNS/nobody/TA-jira-service-desk-simple-addon/ta_service_desk_simple_addon_settings/advanced_configuration -d 'kvstore_instance=acme.splunk.com:8089'

Setting the bearer token¶

When running in client mode, set the bearer token for the remote KVstore access:

curl -k -u admin:'ch@ngeM3' -X POST https://localhost:8089/servicesNS/nobody/TA-jira-service-desk-simple-addon/ta_service_desk_simple_addon_settings/advanced_configuration -d 'bearer_token=xx-xx-xx-xx-xx-xx'

Setting the KVstore filter¶

When running in client mode, optionally set the KVstore filter (for example to dedicate a given client to specific accounts):

curl -k -u admin:'ch@ngeM3' -X POST https://localhost:8089/servicesNS/nobody/TA-jira-service-desk-simple-addon/ta_service_desk_simple_addon_settings/advanced_configuration -d 'kvstore_search_filters=(account="*")'

Setting the logging mode¶

To enable DEBUG logging:

curl -k -u admin:'ch@ngeM3' -X POST https://localhost:8089/servicesNS/nobody/TA-jira-service-desk-simple-addon/ta_service_desk_simple_addon_settings/logging -d 'loglevel=DEBUG'

Enable and configure the proxy¶

Example:

curl -k -u admin:'ch@ngeM3' -X POST https://localhost:8089/servicesNS/nobody/TA-jira-service-desk-simple-addon/ta_service_desk_simple_addon_settings/proxy -d 'proxy_enabled=1' -d 'proxy_url=myproxy.domain.com' -d 'proxy_port=8080'

Additional options are:

proxy_username (string)
proxy_password (string)
proxy_rdns (boolean, 0 disabled, 1 enabled)
proxy_type (http/socks4/socks5)

Advanced configuration and more information¶

JIRA instance configuration and authentication¶

The Add-on for JIRA currently supports two modes of authentication, basic authentication and Personal Access Token authentication.

Basic authentication:

Personal Access Token authentication (PAT):

https://confluence.atlassian.com/enterprise/using-personal-access-tokens-1026032365.html

noauth2 is not yet supported by the Add-on for JIRA, but this integration is under progress

About SSL certificate validation¶

SSL usage is enforced, this means you cannot access to a JIRA instance if it is not using SSL.

When using SSL, there might be different conditions:

The SSL certificate is from an official third party certificate authority and it is valid, you normally can tick the SSL validation box which enforces the fact that we expect the certificate to be valid. If the certificate is not valid, we will not allow any action to be performed.
The SSL certificate is a self-signed certificate, you cannot verify the SSL certificate as it not a valid certificate, therefore the SSL validation box must not be ticked.
The SSL certificate is from an internal PKI, it is valid but not trusted by default by the operating system, you can use the SSL certificate path to specifiy the local path to the corresponding certificate bundle and tick the validation box. If the file exists, it will be used during the REST calls, otherwise the SSL validation will be ignored.

For more information about validating an internal certificate: https://docs.python-requests.org/en/stable/user/advanced/#ssl-cert-verification

Logging level¶

The logging level can be defined within the configuration page too, the application makes a real usage of the debug mode and will generate many more messages in debug.

In normal circumstances, the logging level should be defined to INFO, required logging level will automatically be used when any unexpected error is encountered.

Validating the connectivity¶

You can validate the connectivity very easily by opening any of the JIRA Get information reports, which achieve rest calls to the JIRA API to retrieve different information such as the list of projects available:

Shall the connectivity be effective and if you open the Get projects report, the list of the JIRA projects available for your JIRA instance appears in the table.

| jirafill opt=1 | stats count by key, key_projects

If the command returns the list of your JIRA projects per account configured, then the connectivity is successful:

You can as well simulate the creation of an alert and action the JIRA Service Desk:

Enter a search window
type |makeresults
Click save as new alert
Scroll down to alert actions and add the JIRA Service Desk action

Testing access and authentication with curl:

You can as well very easily achieve a test with curl from the search head:

With basic authentication:

curl -k https://<jira_url>/rest/api/latest/project --user <jira_username>:<jira_password>

With PAT authentication:

curl -H "Authorization: Bearer <yourToken>" https://<jira_url>/rest/api/latest/project

Which, if successful, will return in a JSON format the list of projects available in your JIRA instance.

Using the alert action for non admin users¶

For non admin users to be able to use the alert action, the following role is provided out of the box:

jira_alert_action

This role needs to be inherited for the users, or your users to be member of this role.

The role provides:

capability list_storage_passwords
capability list_settings
write permission to the resilient KVstore kv_jira_failures_replay
wirte permission to the backlog KVstore jira_issues_backlog

Distributed setup (passthrough mode)¶

What is the JIRA passthrough?¶

The passthrough has been designed for specific use cases where the Splunk main deployment is not capable of reaching directly the JIRA instance due to network and security constraints.

In this scenario, the Search Head layer cannot contact JIRA directly, and we need an on-premise Splunk component to be able to perform the interaction with JIRA, while getting knowledge of what has to be done.

This distributed setup relies on the Splunk KVstore as the intermediate link between the Search Heads requesting an issue to be created, and a Splunk Heavy forwarder backend which will be responsible for its creation:

Configuration UI:

This use case is common enough for Splunk Cloud customers running JIRA on-premise, due to security considerations, it may be refused or complex to open a connectivity between Splunk Cloud and the on-premise JIRA.

Hint

The JIRA passthrough requires a Splunk Heavy Forwarder running on-premise
The Heavy Forwarder needs to be able to access to the Splunk Search Head splunkd API which can be requested to Splunk Cloud Ops
Work with Splunk Cloud teams and potentially Splunk Professional Services to get the setup ready
The final setup will allow JIRA issues creation from alerts (correlation seaches in Enterprise Security) and ad-hoc adaptive response actions in incident review
In passthrough mode, the CSV/JSON attachment feature is not available
In passthrough mode, the JIRA dedup and auto-comment feature is not available

In a nutshell:

The Splunk Cloud search head creates content in a local replay KVstore
The Splunk on-premise Heavy Forwarder automatically accesses the remote KVstore on Splunk Cloud via a bearer authentication
The Heavy Forwarder interacts with JIRA to perform the issues creation, and updates KVstore records accordingly

Using the passthrough mode can accomodate this scenario with some additional configuration and setup, things will work as:

The Add-on is deployed to the Splunk Cloud Search Head(s)
The passthrough mode is enabled on the Splunk Cloud Search Head(s)
The Add-on is deployed to the on-premise Heavy Forwarder
In the Heavy Forwarder, you configure the JIRA accounts with the proper connectivity, and sets the remote KVstore access (URL and bearer token)
In the Splunk Cloud Search Head(s), create the account reference equally, there is no need to setup the account (URL, etc) besides the same account name creation as in the Heavy Forwarder

Step 1: Get the JIRA Add-on installed¶

The JIRA Add-on must be installed to both the Splunk Cloud search, and the on-premise Heavy Forwarder.

Step 2: Splunk Cloud - create the account reference(s) in the Add-on and enable the passthrough¶

To accept creating records in the local replay KVstore, you first need to setup the account reference(s) in the JIRA Add-on.

The only information that needs to be setup is the account name, which needs to match between the Search Head(s) and the Heavy Forwarder.

Example:

Step 3: Splunk Cloud - create a bearer token for the authentication of the Heavy Forwarder API calls¶

Once the Splunk API has been opened by Cloud Ops, you need to create an authentication bearer token that will be used by Add-on on the Heavy Forwarder.

Go in Splunk settings menu:

Settings / USERS AND AUTHENTICATION / Tokens

For reference:

https://docs.splunk.com/Documentation/Splunk/latest/Security/UseAuthTokens

You can decide to create a specific user for this integration, the user needs the relevant permissions on the KVstore. (you can inherit the jira_alert_actions role for this)

Step 4: Heavy Forwarder - configure the real JIRA account(s)¶

Setup the JIRA account(s) with the same exact names as the account(s) that were created in the Search Header:

Note: do not enable the passthrough mode in the Heavy Forwarder!

This is the real JIRA account configuration, which includes URl, crendentials and SSL related configuration, example with PAT:

Step 5: Heavy Forwarder - configure the remove KVstore collection¶

In the Add-on configuration UI, setup the remote KVstore URL and the bearer token:

The KVstore URL is in the form <address:port>, example: acme.splunkcloud.com:8089
The bearer token is the full token value defined in the previous step

Test the connectivity:

On the Heavy Forwarder, run the following custom command:

| getjirakv verify=True

If the connectivity is successul, an HTTP 200 error code is returned as follows:

If the authentication fails, the following message would be returned:

If the remote splunk API cannot be reached:

Multiple Heavy Forwarders setup¶

You certainly had noticed an addition setting called “KVstore search filters” which by default equals to (account="*"):

The purpose of this option is to provide additional capabilities in the distributed setup, such that you could have different environment connected to different JIRA instances via different Heavy Forwarders.

A single Heavy Forwarder can handle any number of JIRA instances (via the multi accounts setup), however it can be required for any reason (network, environments, etc) that additional accounts would be handled by additional Heavy Forwarders.

You can use the KVstore filter to easily and transparently associate a given Heavy Forwarder to specific accounts, and dedicate it according to your needs.

This additional setup could be represented as follows:

By relying on the KVstore filter, you can easily setup any additional Heavy Forwarder and dedicate each instances to handle specific JIRA accounts.

Final review¶

Congratulations! The step is now terminated, as the Heavy Forwarder is forwarding its own internal data to Splunk Cloud indexers, transactions logs are transparently available within the JIRA Add-on UI:

For instance, the JIRA issues “created” on the Splunk Cloud search head, will appear in the first tab and tagged as info:

The logs exposing the real creation of the issues via the replay KVstore are available in the second tab called “Resilient store activity”:

The configuration is now over and fully functional, the Heavy Forwarder honors the normal TA workflow, issues to be created will be removed automatically from the replay KVstore upon a successful creation.

User guide:¶

User guide¶

Using the JIRA Service Desk alert action from alerts and correlation searches¶

Whenever you create or configure a Splunk core alert or Enterprise Security correlation search, you can now select the JIRA Service Desk action to automatically create a new JIRA issue based on the results of a search.

The configuration of the alert is pretty straightforward and described in detail in the further sections of the above documentation.

Using the JIRA Service Desk alert adaptive response action from Splunk Enterprise Security¶

In Splunk Enterprise Security, the JIRA action can be triggered as an adaptive response action from Incident Review:

The same options are available with the same level of features; however, tokens expansion will depend on the notable event context.

JIRA project¶

Several projects might have been created in your JIRA instance; you can choose any of the projects available on per alert basis.

The list of JIRA projects made available within the configuration screen is the result of a dynamic REST call achieved against your JIRA instance anytime you access this screen, which can be reproduced manually too:

| jirafill opt=1 | stats count by key, key_projects

JIRA issue type¶

The type of issue to be created is a dynamic list provided by JIRA based on the types available for the project that has been selected, these are the result of the following command:

| jirafill opt=2 | stats count by issues

JIRA issue priority¶

The priority of the issue is dynamically retrieved from the JIRA project based on the different priorities that are made available by your JIRA screen configuration, these are the results of the following command:

| jirafill opt=3 | stats count by priorities

JIRA issue dynamic priority¶

The dynamic priority is a feature that allows you to dynamically define the priority based on the search result rather than a selected priority from the dynamic list provided by JIRA.

To use the priority of the search results, you need to define a field in your search results that exactly match the priority value expected by JIRA, which can obviously be the results of conditional operations in your SPL logic.

Assuming the following simplistic example in your search:

| eval jira_priority=case(count<10, "low", count>=10 AND count<50, "medium", count>=50, "high")

You will define the dynamic priority to: $result.jira_priority$

The dynamic priority is entirely optional and is only used if it has been defined in the alert configuration.

JIRA summary and description¶

JIRA summary and description are the core information of a JIRA issue.

These two fields define the title of the JIRA issue, and its main content visible to your JIRA users.

Both fields will automatically handle any dynamic value that are available from the results of your search, which requires to be defined as $result.myfield$ to be automatically translated into the relevant value.

JIRA assignee¶

The JIRA assignee field is optional, and can be defined to a static or a dynamic value (using a token) to automatically assign the issue to a specific JIRA user.

JIRA reporter¶

The JIRA reporter field is optional, and can be defined to a static or a dynamic value (using a token) to automatically assign the issue to a specific JIRA user.

JIRA labels¶

JIRA labels is an optional field, which can be defined as a comma separated list of values to assign a list of labels to the JIRA issue.

JIRA components¶

JIRA components is an optional field, which can be defined as a comma separated list of values to assign a list of components to the JIRA issue. (by their names)

JIRA dedup behavior¶

The JIRA deduplication is a powerful feature that allows to automatically control the decision to create or update an issue, which relies on a bidirectional integration with JIRA.

The feature relies on 3 main options:

JIRA dedup behaviour: this enables the dedup feature, disabled by default
JIRA dedup excluded status categories: A comma seperated list of statuses that will be considered for the decision
JIRA dedup content: (Optional) Provides extra control on the content used to make the decision

Let’s take the following example to explain how the feature works:

The following search simulates an alert triggering:

| makeresults
| eval user="foo@splunk.com", action="failure", reason="Authentication failed"
| eval time=strftime(_time, "%c")

everytime the alert triggers, the values for user, action and reason remain the same
the time value differs every time the action triggers

Let’s enable the JIRA alert action, we’ll include in the description field all the fields from resulting from the alert:

For now, we didn’t enable the dedup feature, if we use the DEBUG logging mode, the logs will show the full JSON payload sent to the JIRA API in pretty print manner:

Use the navigation bar shortcut to access the logs, the final JSON is logged with a message: json data for final rest call

Even if we didn’t enable yet the feature, the Addon calculates an MD5 sum which is recorded in a KVstore collection, traces are logged about this:

2021-06-25 20:33:05,394 DEBUG pid=5759 tid=MainThread file=cim_actions.py:message:243 | sendmodaction - signature="jira_dedup: The calculated md5 hash for this issue creation request (db05a46bd3a2e6ccb57906cd749db047) was not found in the backlog collection, a new issue will be created" action_name="jira_service_desk" search_name="Test JIRA - demo dedup" sid="scheduler__admin__search__RMD526ad4cfa87997743_at_1624653180_13" rid="0" app="search" user="admin" action_mode="saved"

The MD5 sum is calculated against the entire JSON data.

To access the KVstore collection containing these records, look at the nav menu “KVstore collections / JIRA Service Desk - Issues backlog collection”.

As every ticket corresponds to a new issue, the status is “created”.

Now, let’s modify a bit the alert, we will remove the time field from the description in JIRA, and enable the dedup:

As the content of the JSON is exactly the same (we removed the time from the description), the Addon will detect it and perform an update of first created issue, adding a comment, and updating the record in the KVstore lookup:

2021-06-25 20:45:06,360 INFO pid=8814 tid=MainThread file=cim_actions.py:message:243 | sendmodaction - signature="jira_dedup: An issue with same md5 hash (60727858c049e599fdb68a3cd744a911) was found in the backlog collection, as jira_dedup is enabled a new comment will be added if the issue is active. (status is not resolved or any other done status), entry:={ "jira_md5" : "60727858c049e599fdb68a3cd744a911", "ctime" : "1624652826.254012", "mtime" : "1624652826.2540202", "status" : "created", "jira_id" : "10100", "jira_key" : "LAB-76", "jira_self" : "https://localhost:8081/rest/api/2/issue/10100", "_user" : "nobody", "_key" : "60727858c049e599fdb68a3cd744a911" }" action_name="jira_service_desk" search_name="Test JIRA - demo dedup" sid="scheduler__admin__search__RMD526ad4cfa87997743_at_1624653900_33" rid="0" app="search" user="admin" action_mode="saved" action_status="success"

The KVstore collection shows a status “updated” for the issue:

The Addon UI shows as well that updates were performed rather than new issues creation:

The issue itself in JIRA shows new comments added everytime the alert triggered for the same content:

We can control the content of the comment added to the issue by creating a custom field in the resulting Splunk alert, let’s modify the alert to include a new field used to control the comment:

| makeresults
| eval user="bar@splunk.com", action="failure", reason="Authentication failed"
| eval time=strftime(_time, "%c")
| eval jira_update_comment="The same condition was detected by Splunk for the user=" . user . " with action=" . action . " and reason=" . reason . ", therefore a new comment was adeed to the JIRA issue."

After the first issue creation, the next time the alert triggers, the Addon will use the content of the “jira_update_comment” field and use in the comment field in JIRA:

Issue initially created:

Issue updated with our comment field:

Now, let’s say this issue is taken in charge in JIRA, it status is changed to Done as we think the underneath condition is fixed:

This is where the second dedup option acts, thanks to this bi-directional integration, the Addon knows that the issue was fixed and decides to open a new issue.

An INFO message is logegd explaining why the Addon took this decision:

2021-06-26 09:42:06,237 INFO pid=13894 tid=MainThread file=cim_actions.py:message:243 | sendmodaction - signature="jira_dedup: The issue with key LAB-109 has the same MD5 hash: 60727858c049e599fdb68a3cd744a911 and its status was set to: "Done" (status category: "Done"), a new comment will not be added to an issue in this status, therefore a new issue will be created." action_name="jira_service_desk" search_name="Test JIRA - demo dedup" sid="scheduler__admin__search__RMD526ad4cfa87997743_at_1624700520_67" rid="0" app="search" user="admin" action_mode="saved" action_status="success"

If you have custom statuses, you can update the list of statuses to be taken into account in the alert definition, the Addon accepts a comma separated list of statuses.

Now, let’s say that we need to have more information added into our JIRA ticket, some will not change if the same alert triggers for the same condition, but others that we need such as the time field will always differ.

To achieve our goal, we will use the third option to “scope” what the Addon will use for the MD5 generation that is used to idenfity a duplicate issue, we will generate a specific field in the Splunk alert and recycle its value in the alert definition:

| makeresults
| eval user="foo@splunk.com", action="failure", reason="Authentication failed"
| eval time=strftime(_time, "%c")
| eval jira_update_comment="The same condition was detected by Splunk for the user=" . user . " with action=" . action . " and reason=" . reason . ", therefore a new comment was adeed to the JIRA issue."
| eval dedup_condition = "user=" . user . "|action=" . action . "|reason=" . reason

Then, we modify our alert action to ask the Addon to use this token variable for the MD5 generation:

note: $result.dedup_condition$ is how you will instruct Splunk to recycle dynamically the value of the field dedup_condition and pass it in the alert action.

We have now changed the way we idenfity what is a duplicate, and what is not, we can have fields which content will always change like our time field without breaking the dedup idenfitication:

When the alert triggers more than once, we can see a new comment added to our issue:

The same workflow applies again, if we fix the issue the Addon will detect it and create a new ticket, if something happens to be different in the condition for the dedup idenfitication, a new ticket will be created.

Powerful, isn’t?!

Additional information about the KVstore knowledge records:

key is the internal uuid of the KVstore, as well the key will be equal to the md5 hash of the first occurrence of JIRA issue created (next occurrences will have a key uuid generated automatically with no link with the md5 of the issue)
ctime is the milliseconds epochtime that corresponds to the initial creation of the ticket, this value can not be changed once the record is created
mtime is the milliseconds epochtime of the last modification of the record, if a comment is added to this ticket, this value corresponds to the time of that action
jira_md5 is the actual md5 hash for the entire JIRA issue, when the dedup option is activated for an alert, this will always be equal to the key id of the record in the KVstore
status reflects the status of the issue as it is known from the add-on perspective, created means the issue was created, updated means at least one comment was made to this ticket due to dedup matching
jira_id / jira_key / jira_self are JIRA information related to this ticket

JIRA attachment¶

On a per alter basis, the results from the Splunk alert that triggered can automatically be attached to the JIRA issue.

Features and limitations:

The attachment feature is disabled by default, and needs to be enabled on a per alert basis
The format of the results can be attached in CSV format, JSON or XLS (Excel) format
The feature is not compatible with the resilient store, if the JIRA issue initially fails due to a temporary failure, the ticket will be created by the resilient tracker when possible but without the original attachment

When the attachment option is enabled, the following message will be logged if the attachment was successfully added to the JIRA issue, in addition with details of the ticket returned by JIRA:

JIRA Service Desk ticket attachment file uploaded successfully

File attachment in JIRA:

Note: the file name is dynamically generated, prefixed with “splunk_alert_results_” and suffixed by the relevant file extension.

JIRA custom fields¶

JIRA custom fields are fields that can designed by your JIRA administrators to be available during the issue creation.

The Splunk Add-on for JIRA Service Desk supports any kind and any number of custom fields by allowing you to insert a custom field JSON structure in the alert configuration.

There are different types of custom fields, from a single ling text input to date and time pickers, which are described in the JIRA API documentation:

https://developer.atlassian.com/server/jira/platform/jira-rest-api-examples

Depending on the format of the custom field, you need to use the proper syntax, the most common are:

"customfield_10048": "$result.singleline_text$",

"customfield_10052": {"value": "$result.single_choice$"},

"customfield_10053": [ {"value": "$result.multi_choice_grp1$" }, {"value": "$result.multi_choice_grp2" }]

As usual, while you define the custom fields, you can use dynamic results from the Splunk search results by using the syntax: $result.myfield$

To add a list of custom fields, make sure you add a comma after each custom field, and none at the end of the JSON structure.

A full example JSON structure is provided in the alert action screen:

"customfield_10048": "$result.singleline_text$",
"customfield_10052": {"value": "$result.single_choice$"},
"customfield_10053": [ {"value": "$result.multi_choice_grp1$" }, {"value": "$result.multi_choice_grp2" }]

Custom fields parsing:

By default, the content of the custom fields is parsed to escape and protect any special characters that would potentially lead the JSON data not to be parsed properly.

In some circumstances, the built-in parser rules may fail to recognize an unexpected custom fields structure, the parsing can be disabled if required:

How to retrieve the IDs of the custom fields configured ?¶

Use the built-in report and associate custom command to retrieve the list of JIRA fields information:

This report achieves a REST call to JIRA to get the list of fields and their details per project and per type of issues, search for custom fields:

JIRA REST API wrapper¶

A custom command is provided as a generic API wrapper which can be used to get information from JIRA by calling any REST endpoint available: By default, it uses method GET. Additional methods are supported DELETE, POST, PUT.

| jirarest target="<endpoint>"

Open the REST API dashboard to get examples of usage:

The following report is provided to retrieve issues statistics per project and per status categories:

JIRA Service Desk - Issues statistics report per project

Indexing JIRA statistics for reporting purposes¶

If you wish to index the JIRA statistic results in Splunk for reporting purposes over time, you can easily modify or clone this report to use collect or mcollect to index these statistics:

Indexing the results to a summary report¶

You can use the collect command to automatically index the report results in a summary index of your choice, schedule this report and add a call to collect, example:

| collect index=summary source="JIRA - issues stats per project"

Indexing the results to a metric index¶

Another option is to use the mcollect command to automatically index these statistics as native metrics in a metric index of your choice, the following example assumes a metric index named “jira_metrics” was created, the report scheduled and the following mcollect command is added:

| eval type="jira_" | mcollect split=t prefix_field=type index=jira_metrics project

Each statistic is stored as a metric_name with a prefix “jira_”, while the project is stored as a dimension, you can use the mcatalog and mstats commands to use the metrics, or use the Analytics view in Splunk:

mcatalog example:

| mcatalog values(metric_name) values(_dims) where index=jira_metrics metric_name=jira_*

mstats example:

| mstats latest(jira_pct_total_done) as pct_total_done, latest(jira_pct_total_in_progress) as pct_total_in_progress, latest(jira_pct_total_to_do) as pct_total_to_do where index=jira_metrics by project span=5m

Additional examples for JIRA API wrapper¶

Method DELETE: Delete an issue¶

| jirarest target="rest/api/2/issue/{issueIdOrKey}" method=DELETE

Method POST: Add a comment to an issue¶

Example 1:

| jirarest target="rest/api/2/issue/{issueIdOrKey}/comment" method=POST json_request="{\"body\": \"This is a normal comment.\"}"

Example 2:

| jirarest target="rest/api/2/issue/{issueIdOrKey}/comment" method=POST json_request="{\"body\": \"This is a comment that only administrators can see.\", \"visibility\": {\"type\": \"role\", \"value\": \"Administrators\"}}"

Method PUT: Assign an issue¶

| jirarest target="rest/api/2/issue/{issueIdOrKey}/assignee" method=PUT json_request="{\"name\": \"harry\"}"

Troubleshoot:¶

Trouble shooting¶

Connectivity to JIRA issues¶

If the connectivity to JIRA is not valid for some reasons (bad credentials, network connecttivity, etc), this will result in different Python error messages when attempting to load any of the report, load the alert action page or execute an alert action.

In such as case, the easiest way is to validate the connectivity by achieving a rest cal using the curl command in CLI, ideally in any of the search head supposed to be using the alert action. (note: this step is valid for Linux only)

curl -k https://<jira_url>/rest/api/latest/project --user <jira_username>:<jira_password>

For more information, follow these links:

Overview dashboard and Add-on logs¶

The Splunk Add-on for JIRA Service Desk provides a builtin Overview dashboard that gives deep insights on the Add-on activity:

The dashboard exposes the JIRA issue workflow and direct links to access the Add-on logs.

Add-on logs for first REST call attempts¶

When the alert action is triggered, the Add-on records its activity in:

(index="_internal" OR index="cim_modactions") (source="*jira_service_desk_modalert.log")

When the JIRA issue is successfully achieved, the key sentence JIRA Service Desk ticket successfully created is logged.

If an error is encountered during the API call, the key sentence JIRA Service Desk ticket creation has failed is logged.

When the failure step is reached, for example if there is an issue with the credentials or reaching the JIRA instance, the workflow records the failure in a resilient store based on a KVstore lookup:

| inputlookup jira_failures_replay | eval uuid=_key

At this point, any failed call recorded in the KVstore is automatically re-attempted by the scheduled alert named: JIRA Service Desk - Resilient store Tracker

An out of box alert named JIRA Service Desk - detection of temporary issue creation failure is provided to monitor and track any JIRA failure, the alert is by default enabled.

Add-on logs for the resilient store feature¶

The resilient store feature tracks its activity in:

(index="_internal" OR index="cim_modactions") (source="*jira_service_desk_replay_modalert.log")

In normal circumstances, which means there have not been recent failed attempts, there would be no activity in this logs, nor content in the KVstore.

If a record exists in the KVstore, the Add-on will re-attempt the creation every 5 minutes during 3 days per record, if it continuously failed durant that period, a key sentence permanent failure! is logged.

An out of box alert named JIRA Service Desk - detection of permanent issue creation failure is provided to monitor and track permanent JIRA failures, the alert is by default enabled.

After 7 days in the KVstore, a record is automatically and definitively purged.

Root cause for failures¶

Root causes of failures will be clearly exposes in the Add-on logs, most common causes could be:

JIRA credential issues (verify the connectivity, see the configuration page)
Networking issues or JIRA instance not reachable
Content issues such as JIRA fields not available on the JIRA project (make sure these fields are associated with the right JIRA screens)
Content issues such as JIRA field receiving an unexpected content or format (some JIRA fields such as date and date time inputs require a valid format, etc)

Shall a REST call for JIRA issue creation fail, the Add-on automatically logs full JSON data which you can use to easily review the data and trouble shoot the root causes.

Versions and build history:¶

Release notes¶

Version 2.0.2¶

Warning

BREAKING CHANGES!

The new major release uses a new framework (add-on-ucc-framework) which changes the way accounts are handled by the application
Post upgrade, you need to setup the connectivity to your JIRA instance(s) again before the Add-on can be used
Existing alerts will not work anymore until you perform the account setup
You do not need to update the alerts themselves as these remain compatible from version 1.x to version 2.x

What’s new in the Add-on for JIRA version 2.0.x:

Fix - Issue #106 - Windows specific - Addon writing output CSV into WindowsTEMP folder

Version 2.0.1¶

Warning

BREAKING CHANGES!

The new major release uses a new framework (add-on-ucc-framework) which changes the way accounts are handled by the application
Post upgrade, you need to setup the connectivity to your JIRA instance(s) again before the Add-on can be used
Existing alerts will not work anymore until you perform the account setup
You do not need to update the alerts themselves as these remain compatible from version 1.x to version 2.x

What’s new in the Add-on for JIRA version 2.0.x:

Fix Appinspect warning check_reload_trigger_for_all_custom_confs #104

Version 2.0.0¶

Warning

BREAKING CHANGES!

The new major release uses a new framework (add-on-ucc-framework) which changes the way accounts are handled by the application
Post upgrade, you need to setup the connectivity to your JIRA instance(s) again before the Add-on can be used
Existing alerts will not work anymore until you perform the account setup
You do not need to update the alerts themselves as these remain compatible from version 1.x to version 2.x

What’s new in the Add-on for JIRA version 2.0.0:

Migration to ucc-gen (Splunk Add-on factory framework), refreshed modern configuration UI
Support for JIRA multi tenant accounts (Multiple JIRA accounts can now be set up targeting different JIRA instances)
Support for Personal Access Token (PAT) authentication (See: https://confluence.atlassian.com/enterprise/using-personal-access-tokens-1026032365.html)
Support for Proxy authentication
Python 3 only support (Splunk 7.x is not supported any longer)
Jquery migration
Improved distributed setup with bearer based remote KVstore feature relying on the replay KVstore (for setups where JIRA is not available from the main Splunk search heads)
Support for attachments in Excel (xlsx) format
Support for attachments with the issue dedup feature
Fix - Issue #102 - Issue in dedup behaviour when dedup is enabled but the issue was resolved, closed or cancelled

Version 1.0.30¶

Enhancement: Issue #91 - proxy support for jirarest.py and jirafill.py #91 (Author: 8lex)
Enhancement: Issue #92 - provide an SSL certificate path option for internal PKI certificate validation, honour SSL certificate validation in custom commands
Enhancement: Issue #93 - attachments are now supported when using a proxy
Enhancement: Issue #94 - Specify latest rather than static version 2 in API REST calls to allow last API version to be used when available

Version 1.0.29¶

Enhancement: jirarest supports additional method for extended JIRA integration #85 (Author: Rémi Séguy)

Version 1.0.28¶

Change: Issue #83 - Python Upgrade Readiness App complains about ‘outdated Python SDK’

Version 1.0.27¶

Fix: Issue #77 - Error reported in logs when the issue MD5 is equal, the alert continues to trigger and dedup is disabled

Version 1.0.26¶

Feature: Issue #72 - Provides a new mode called passthrough mode, which is designed for scenarios where Splunk cannot contact the JIRA instance directly for security or restrictions purposes (such as Splunk Cloud potentially). A second Splunk instance that can connect to JIRA instance would recycle the replay KVstore content to perform the final call.
Enhancement: Issue #73 - Provides custom search auto description (searchbnf.conf)

Version 1.0.25¶

Change: Issue #70 - Splunk Python SDK upgrade to 1.6.15

Version 1.0.24¶

Feature: Issue #65 - Allows defining the JIRA Issue reporter

Version 1.0.23¶

Fix: Issue #61 - Custom commands now require Python3 mode explicity which with AoB py3 SDK version causes error messages on the indexers #61

Version 1.0.22¶

Fix: For Splunk Cloud vetting purposes, commands.conf needs to specify python3 explicitly

Version 1.0.21¶

Fix: Issue #54 - Appinspect failure due to missing key in spec file
Fix: Issue #55 - Appinspect failure in reports using the jirarest command due to checks attempting to run the run the reports in non JIRA connected environments, causing the map command to return an error
Feaure: Issue #56 - New Overview JIRA analytic view relying on the new jirarest command that allows live REST calls to JIRA and execution of JQL queries #56

Version 1.0.20¶

Fix: Issue #50 - Deduplication Creating One Duplicate After Item Closed #50

Version 1.0.19¶

Feature: Issue #33 - Exclude closed statuses from the JIRA dedup behavior, to prevent deduplicating closed issues, which list can be customised if required (defaults to Closed,Completed,Canceled)
Feature: Issue #34 - Provides granular control against the content to be taken into account for dedup behavior and the md5 calculation used to identify duplicated tickets
Feature: Provide a new REST API custom command wrapper to allow performing any get call against any endpoint of the JIRA API, provides a builting issue statistic report that can be used with collect/mcollect to index issues statistics, provide a new dashboard exposing the wrapper usage
Feature: Jira get field report split into two reports, one for all projects, one report providing results per project
Fix: Issue #41 - Incident Review Manual AR Issue #41
Fix: default.meta does not define permissions for the builtin jira_admin role for the JIRA issue backlog collection used for the dedup feature
Change: Issue #42 - Removing Priority as a Required Input #42
Change: Improved rendering of options and clearness for required inputs in the alert definition
Change: Issue #16 - Deprecation of jiragetfields custom command, which is replaced with calls to the new REST wrapper jirarest

Version 1.0.18¶

Fix: ensure aob configuration replicates in shc environment

Version 1.0.17¶

feature: Enable / Disable custom fields structure parsing new alert option, disabling the custom fields parsing can be useful when the backend fails to parse properly a custom fields structure that is not expected

Version 1.0.16¶

fix: Splunk Cloud vetting refused due to a remaining https protocol check in jiragetfields.py, checking if the URI contains https rather than starts with https

Version 1.0.15¶

fix: Splunk Cloud vetting refused due to https protocol verification checking if the URI contains https rather than starts with https
fix: JIRA dedup feature might under some systems be generating a different hash for the same issue due to a different order of the json data after json load operation in Python, perform the md5 calculation before calling json load

Version 1.0.14¶

fix: remove the automatic addition of the result link in the description field as it systematically creates a different JIRA content, which creates confusion with the dedup JIRA option
fix: change in configuration app the sentence “JIRA token password” to “JIRA password” to avoid confusion between basic authentication and OAuth2 which isn’t used by the Add-on
fix: in some custom configuration, the custom command jiragetfields would not return the expected results, the type of issue is removed from the rest call to retrieve all fields information on a per project basis instead

Version 1.0.12¶

Feature: Issue #18 - New option on a per alert basis allows automatically attaching Splunk alert results to the JIRA issue in CSV or JSON format
Feature: Issue #18 - Add by default in the description field the result link token call

Version 1.0.11¶

Feature: Issue #12 - New JIRA deduplication feature workflow allows handling automatically on a per alert basis updating JIRA issues by the addition of a comment (that can be controlled) to the original issue, instead of creating duplicated JIRA issues
Feature: Issue #15 - Adding support for components definition on a per alert basis, components can now be defined by their name in a comma separated format within alerts
Feature: Upgrade of Jinja2 2.11.2 libraries to address vulnerabilities reported during Splunk Cloud app vetting process
Feature: Upgrade of PyYAML 5.3.1 libraries to address vulnerabilities reported during Splunk Cloud app vetting process
Feature: Upgrade of httplib2-0.18.1 libraries to address vulnerabilities reported during Splunk Cloud app vetting process
Feature: Upgrade of urllib3-1.25.9 libraries to address vulnerabilities reported during Splunk Cloud app vetting process

Version 1.0.10¶

Fix: Issue #9 - Parsing failure in custom field section with non standard fields in between square brackets

Version 1.0.9¶

Fix: Issue #11 - SSL verification disablement is not honoured properly and remains active even if the checkbox is not checked
Change: app.manifest schema upgrade to 2.0.0 to ease Cloud automated deployments

Version 1.0.8¶

Fix: Allows defining non custom fields in the custom section, such as builtin non standard fields (Components) that would have been made required by JIRA admins

Version 1.0.7¶

Fix: Default timed out value during REST calls are too short and might lead to false positive failures and duplicated creation of JIRA issues

Version 1.0.6¶

Change: For Splunk Cloud vetting purposes, explicit Python3 mode in restmap.conf handler

Version 1.0.5¶

Fix: Provide an embedded role jira_alert_action that can be inherited for non admin users to be allowed to fire the action and work with the resilient store feature

Version 1.0.4¶

Feature: resilient store improvements, catch all failures and exceptions during issue creation attempts
Fix: minor fix in resilient store table
Fix: remove redundant alert link in nav bar

Version 1.0.3¶

Fix Issue #2: Avoids error messages on indexers in distributed mode to report error messages on jirafill and jiragetfields custom commands due to enabled distributed mode
Fix Issue #2: Avoids error messages reported during execution of jirafill and jiragetfields custom commands related to insecure HTTP calls with urllib3

Version 1.0.2¶

Feature: Support for Web Proxy
Feature: Full support for Python 3 (migration to newer Add-on builder libs, embedded custom commands)
Fix: Support defining the JIRA instance URL with or without https://
Fix: Potential creation failure with number type custom fields
Fix: Metadata avoid sharing alerts, reports and views at global level
Fix: Help block appears right shifted within Enterprise Security correlation search editor, but centered properly in Splunk core alert editor

Version 1.0.1¶

unpublished

Version 1.0.0¶

initial and first public release

Welcome to the Splunk Add-on for JIRA Atlassian Service Desk application documentation¶

Overview:¶

About¶

Compatibility¶

Splunk compatibility¶

Splunk Enterprise Security compatibility¶

Python support¶

Web Browser compatibility¶

JIRA compatibility¶

Support & donate¶

Splunk Community¶

Splunk community slack¶

Open a issue in Git¶

Email support¶

Download¶

Splunk base¶

GitHub¶

Deployment and configuration:¶

Deployment & Upgrades¶

Deployment matrix¶

Dependencies¶

Initial deployment¶

Upgrades¶

Upgrade from version 1.x.x to 2.x.x¶

Configuration¶

Configuring in Splunk Web¶

JIRA account configuration¶

Configuring an account with Personal Access Token (PAT)¶

Configuring an account with basic authentication¶

Configuring via REST API¶

Configuring the JIRA instance via curl¶

Enabling the passthrough mode¶

Setting the KVstore instance and port¶

Setting the bearer token¶

Setting the KVstore filter¶

Setting the logging mode¶

Enable and configure the proxy¶

Advanced configuration and more information¶

JIRA instance configuration and authentication¶

About SSL certificate validation¶

Logging level¶

Validating the connectivity¶

Using the alert action for non admin users¶

Distributed setup (passthrough mode)¶

What is the JIRA passthrough?¶

Step 1: Get the JIRA Add-on installed¶

Step 2: Splunk Cloud - create the account reference(s) in the Add-on and enable the passthrough¶

Step 3: Splunk Cloud - create a bearer token for the authentication of the Heavy Forwarder API calls¶

Step 4: Heavy Forwarder - configure the real JIRA account(s)¶

Step 5: Heavy Forwarder - configure the remove KVstore collection¶

Step 6: Splunk Cloud - create lookups to populate the alert action dropdown¶

Multiple Heavy Forwarders setup¶

Final review¶

User guide:¶

User guide¶

Using the JIRA Service Desk alert action from alerts and correlation searches¶

Using the JIRA Service Desk alert adaptive response action from Splunk Enterprise Security¶

JIRA project¶

JIRA issue type¶

JIRA issue priority¶

JIRA issue dynamic priority¶

JIRA summary and description¶

JIRA assignee¶

JIRA reporter¶

JIRA labels¶

JIRA components¶

JIRA dedup behavior¶

JIRA attachment¶

JIRA custom fields¶

How to retrieve the IDs of the custom fields configured ?¶

JIRA REST API wrapper¶

Indexing JIRA statistics for reporting purposes¶

Indexing the results to a summary report¶

Indexing the results to a metric index¶

Additional examples for JIRA API wrapper¶

Method DELETE: Delete an issue¶

Method POST: Add a comment to an issue¶

Method PUT: Assign an issue¶

Troubleshoot:¶

Trouble shooting¶