Can we synchronize projects from 2 private JIRA server using intermediate Cloud instance

General Questions
,
1

Originally asked by Hung Nguyen on 10 September 2019 (original question)

The goal is to synchronize issues from 2 projects from 2 private JIRA servers. One can not see the other, but each one can have access to the Internet.

One of the possible option I could think of is to use an intermediate site, which could be a JIRA Cloud instance, accessible by both private JIRA, and using Exalate to synchronize issues among them:

  • Issue created Server A, meeting certain criteria, will be sent to Cloud instance, and create a mirror issue here
  • Then Cloud instance sees that this newly created issue has certain criteria that needs to be sent to server B will sent the information to server B to create a peer issue there
  • Update from A or B will be sent to Cloud instance and redirected to the other.

In reality configuration, the above steps don’t work, since in Step 2, Exalate in the Cloud instance saw that the issue is newly created by Exalate itself, so it will ignore its own change, and doesn’t send the request to server B

Is there any work around for this? For ex:

  • If we can say that the proxy user for the 1st connection is proxy1, and for 2nd connection is proxy2, the issue created in Cloud by proxy1 can still be handled subsequently by proxy2 as a different user, hence allowed to be sent to server B
  • Or we can have some way to tell Exalate that the 2 connections are different so can be relayed the requests just fine.

Do you have any advice on this or any other option to achieve the goal?

2

Answer by Francis Martens (Exalate) on 21 September 2019

It took a while to resolve this one. The script listener API of Script Runner for Jira Cloud can be improved. If you want to have the details of the research check the Atlassian development community article here.

It boils down to testing if the event is a jira:issue_updated event, and then test that the update has been performed by Exalate (using the available User object). All other events can pass as they will not create the dreaded loop.

The snippet has been updated

here

Let me know how it goes

Comments:

Hung Nguyen commented on 26 September 2019

Hi Francis,

I have successfully tested the simple connection with 2 separate listeners, one for Comments and another for Issue Created/Updated. And things seemed to work.

I will need to try your suggestion too to consolidate the listeners. But it least the solution works and I have demo-ed to the team about this. I think they like it.

Of course, if Exalate can have something to simplify this then it would be even better.

Thanks for your help.

Francis Martens (Exalate) commented on 26 September 2019

We are thinking of delivering a docker image containing the logic for configuring multiple ‘channels’, each channel acting as a tunnel to connect a pair of nodes.

The container can be deployed anywhere where 2 instances can meet.

Hung Nguyen commented on 02 November 2019

Hi Francis,

After we tested more on this, there are some unexpected results occurred from this setup:

  • If both end-sides are updating on different fields of the same issues (corresponding issue for that site), then the middle site will eventually clear out the updated field by the non-updated value from the other side
Hung Nguyen commented on 02 November 2019

Francis,

This approach has shown a loop back to the originator site and caused error.

A does a transition from Status 1 to Status 2, sync-ing to B to do the same thing.

While it’s doing it, A continues to transition from Status 2 to Status 3,

But at that time B is just finishing the first transition from Status 1 to Status 2, it triggers a Time update by Script runner, which then send back Status 2 back to A, forcing A to move back from Status 3 to Status 2. Which is NOT what we want

Francis Martens (Exalate) commented on 07 November 2019

Back to the drawing board.

Need to do some head-scratching there.

Francis Martens (Exalate) commented on 15 November 2019

Sorry for the delay here. We have been had a number of debates how to solve this one. It seems we have something workable, which still needs to be spiked out.

Hopefully, I can come back to it in the course of next week.

Hung Nguyen commented on 18 November 2019

We are looking forward to a solution for this problem.

Right now, I had to temporarily keep a flag to say which side is changing the status last. Then prevent that side from updating the Status by exalate.

I hope that you can have a more fool-proof solution.

Francis Martens (Exalate) commented on 18 November 2019

It is the same concept but then based on the change history.

The replica has a time stamp, and you can test if the field changed.

We are delivering an externalised script with a method ifNoConflict

to be used as

ifNoConflict(issue, replica,"status") {
    // put your status logic here
}

Would this fit?

Hung Nguyen commented on 12 December 2019

Hello Francis,

I am not sure I understand the solution here.

What status logic should I put into the function. And how the function is used?

Francis Martens (Exalate) commented on 15 December 2019

Hung Nguyen

I published a small piece regarding conflict handling on the documentation site
‘How to avoid conflicts’ This conflict avoidance approach is based on an update map which is calculated from the change history.

To port this solution to your specific situation, additional steps need to be taken, where the change history needs to be copied over from one side to the other.

A POC has been established. Please reach out at your convenience, and we can discuss the details during a demo.

Hung Nguyen commented on 16 December 2019

Yes, I’d like to see how the PoC works and how we can incorporate that into our case.

Please let us know what would be the next steps?

Hung Nguyen commented on 28 October 2020

Just for the sake of others who need the similar solution, Exalate has helped to configure such a 3-node system to synchronize 2 private Jira instances. So if you need this type of setup, Exalate should be a good choice

Francis Martens (Exalate) commented on 28 October 2020

Thanks Hung Nguyen

We are currently working on the exalate gateway which will allow to connect two private instances in a secure way. The Jira cloud approach has a couple drawbacks which will be addressed in the exalate gateway. If interested - send a mail to earlyaccess@exalate.com for getting more information on the feature. We are looking for early adopters to kick the tires.

3

Answer by Francis Martens (Exalate) on 10 September 2019

Hi Hung Nguyen

Thanks for asking!

Can you check the triggers? In the example configuration below, there are 2 triggers to feed the 2 connections. This ensure that whenever an issue is created on the project SCRUM A, that issue ripples through to project ANTWERP, which ripples through to project SCRUM B

957×224 18.8 KB

Comments:

Hung Nguyen commented on 10 September 2019

Yes. I knew that. And the triggers work well when I manually update something directly on the clone issue in the middle site.

The requirement is Site A will sent to site B if the Component field is compB. And in the middle Cloud site, we also set the Component of this peer issue to be compB. And we had the trigger that if Component = B, sent it to Site B

When an issue is created in Site A with component = compB, the trigger from A fired up and created a peer issue in middle site C, with component = compB

But the trigger in middle site C does NOT take this issue to send to Site B as expected.

If I modify a comment of this issue in site C, then the trigger starts to fire and creates issue to Site B, meaning that the trigger does work

Hung Nguyen commented on 10 September 2019

Please note that in my middle site, all the issues from both sites are put into the only project, namely P, but with different components. And the triggers 1 will take only component compA to send to A, and compB to create/update to B.

Francis Martens (Exalate) commented on 10 September 2019

Hmm - back to the drawing board then - the trigger setup is on a single instance, so there might be something wrong on the cloud connector.

I will try it for myself and circle back

Hung Nguyen commented on 10 September 2019

The triggers on the middle Cloud site are fired only from a manual update to the issues in its site. They ignore all the updates collected by Exalate from either remote sites. I understand that this is to prevent to send back to the original remote site again its own changes. But in this case, we need to relay the changes instead. So if each connection use a different proxy user from the other, then maybe it works

Hung Nguyen commented on 13 September 2019

Do you have any suggestion Francis?

We are longing for a solution on this.

Thanks

Hung

Hung Nguyen commented on 17 September 2019

Francis,

Is there any update on this issue? We want to see if there is any work around for this. Or should abandon the plan of using this?

Thanks

Hung

Francis Martens (Exalate) commented on 18 September 2019

Hi Hung,

Thanks for pinging me frequently - the discussion got off my radar.

Picking it up again

Francis Martens (Exalate) commented on 18 September 2019

Your analysis is spot on, so we found a workaround (until we can integrate something in the product which does this correctly)

Like always - script runner to the rescue.

  1. Add ScriptRunner for Jira to your cloud environment

  2. Create a custom field ‘SyncTrigger’ and associate to the project

  3. Create a ScriptListener in Script runner and set following fields

    1. Name of the listener: priv-priv
    2. Events - Issue created, updated, commented, worklog … whatever you will be updating
    3. As this user: ScriptRunner add-on user
    4. empty
    5. Test if the exalate user has done the update - else you get a loop
    6. Script listener code itself

    Is available here

  4. Give it a try and let us know

Hung Nguyen commented on 19 September 2019

How can I do step e. Test if the exalate user has done the update ?

I read all the exalate document but could not see where we can get this testing.

Can you help with it too?

Thanks

Hung Nguyen commented on 19 September 2019

I put step e. as 1==1 just to make it passed. But the script didn’t execute, even though the condition seemed to execute passed. The execution history is always empty. I’m not sure if it’s because the script ?

Hung Nguyen commented on 19 September 2019

On the history screen shot, you can see the top condition ran, but not the script content at bottom.

Hung Nguyen commented on 19 September 2019

After several testing, I saw that it’s the following test that was evaluated to false

user.displayName == ‘Exalate’

in the condition

And if we put this in the script content, then Scriptrunner complain that user object can not be valid

Francis Martens (Exalate) commented on 19 September 2019

ugh - strange

When you expand the script context - user is one of the available script context objects - isn’t it?

Anyway

It took me a couple of iterations to get it right, the script behind the link

https://stash.idalko.com/snippets/bacc22614f10440d84c6c5d7ea5d042c

should work.

No need to add a condition, as the test in the begin of the script is avoiding the loop which I encountered when testing

Hung Nguyen commented on 19 September 2019

This is the exact error message in the history event, when I put the if (user.displayName = “Exalate”)

at the top of the script:

2019-09-19 20:59:51.786 ERROR - No such property: user for class: Script1 on line 1
2019-09-19 20:59:52.607 ERROR - Class: com.adaptavist.sr.cloud.events.WebhookExecution, Config: null
Hung Nguyen commented on 19 September 2019

OK, I looked closer into the ScriptRunner document. Only Issue Created and Issue Updated got ‘user’ in context. For Comment created and Comment Updated event, there is no user. Surprise!

Now the workaround looks a bit uglier. And for Attachment Created event, there is even NO issue context. Not sure how can update issue field in that case.

Francis Martens (Exalate) commented on 20 September 2019

Ah - I understand - I will investigate and come back to it.

Francis Martens (Exalate) commented on 26 September 2019

Hung Nguyen - looking forward to your feedback

4

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.