AutomatiCloud

Posted: **Tue Oct 25, 2016 2:58 am**

Hi,

I have AutomatiCloud v3.2.638.2241 running under a purchased license.

I have the 'default' setting set up for region 'ap-southeast-2' (Sydney). Under this setting I have a number of 'Backup EC2 Instance' jobs defined that also copy the snapshots to region 'ap-northeast-1' (Tokyo). These jobs work fine.

However I also have an instance running in 'eu-west-1' (Ireland) that I also want to backup and copy to Tokyo. I've therefore created a 2nd AutomatiCloud setting 'Ireland' and defined a backup job against it otherwise identically to the others.

This job fails to copy snapshots from Ireland to Tokyo.

I get a series of emails from AutomaticCloud as follows. Note that the instance has 3 volumes.

AutomatiCloud Notification (JOBNAME) - success (x3)
EBS snapshot created for volume ID: vol-326d95b0 Snapshot ID:snap-c0cacbf3 Job Type: Backup EC2 Instance Job Name: JOBNAME Job ID: i-f59167c4-201610250848488110
Host: HOSTNAME
Timestamp: 25/10/2016 11:16:33 AM

AutomatiCloud Notification () - failure (x3)
EBS snapshot not copied. Snapshot ID:snap-c0cacbf3 Job Type: Copy EC2 Snapshot Job Name:
Job ID: AWSCopySnapshotJob-201610251116335208
Host: HOSTNAME
Timestamp: 25/10/2016 11:16:44 AM

AutomatiCloud Notification (JOBNAME) - success
Backup Instance ID:i-f59167c4 completed.
Job Type: Backup EC2 Instance
Job Name: JOBNAME
Job ID: i-f59167c4-201610250848488110
Host: HOSTNAME
Timestamp: 25/10/2016 11:17:47 AM

The service logfile for the same, including service startup, is as follow:

2016-10-25 11:15:01,185 [1] INFO Quartz.Util.DBConnectionManager - Registering datasource 'default' with db provider: 'Quartz.Impl.AdoJobStore.Common.DbProvider'
2016-10-25 11:15:01,191 [1] INFO Quartz.Impl.StdSchedulerFactory - Using default implementation for object serializer
2016-10-25 11:15:01,196 [1] INFO Quartz.Impl.StdSchedulerFactory - Using custom data access locking (synchronization): Quartz.Impl.AdoJobStore.UpdateLockRowSemaphore
2016-10-25 11:15:01,196 [1] INFO Quartz.Impl.StdSchedulerFactory - Using default implementation for ThreadExecutor
2016-10-25 11:15:01,205 [1] INFO Quartz.Core.SchedulerSignalerImpl - Initialized Scheduler Signaller of type: Quartz.Core.SchedulerSignalerImpl
2016-10-25 11:15:01,205 [1] INFO Quartz.Core.QuartzScheduler - Quartz Scheduler v.2.2.3.400 created.
2016-10-25 11:15:01,211 [1] INFO Quartz.Impl.AdoJobStore.JobStoreTX - JobStoreTX initialized.
2016-10-25 11:15:01,213 [1] INFO Quartz.Core.QuartzScheduler - Scheduler meta-data: Quartz Scheduler (v2.2.3.400) 'QuartzScheduler' with instanceId 'NON_CLUSTERED'
Scheduler class: 'Quartz.Core.QuartzScheduler' - running locally.
NOT STARTED.
Currently in standby mode.
Number of jobs executed: 0
Using thread pool 'Quartz.Simpl.SimpleThreadPool' - with 10 threads.
Using job-store 'Quartz.Impl.AdoJobStore.JobStoreTX' - which supports persistence. and is not clustered.

2016-10-25 11:15:01,213 [1] INFO Quartz.Impl.StdSchedulerFactory - Quartz scheduler 'QuartzScheduler' initialized
2016-10-25 11:15:01,213 [1] INFO Quartz.Impl.StdSchedulerFactory - Quartz scheduler version: 2.2.3.400
2016-10-25 11:15:01,304 [16] INFO Quartz.Impl.AdoJobStore.JobStoreTX - Freed 0 triggers from 'acquired' / 'blocked' state.
2016-10-25 11:15:01,310 [16] INFO Quartz.Impl.AdoJobStore.JobStoreTX - Recovering 0 jobs that were in-progress at the time of the last shut-down.
2016-10-25 11:15:01,310 [16] INFO Quartz.Impl.AdoJobStore.JobStoreTX - Recovery complete.
2016-10-25 11:15:01,311 [16] INFO Quartz.Impl.AdoJobStore.JobStoreTX - Removed 0 'complete' triggers.
2016-10-25 11:15:01,311 [16] INFO Quartz.Impl.AdoJobStore.JobStoreTX - Removed 0 stale fired job entries.
2016-10-25 11:15:01,323 [16] INFO Quartz.Core.QuartzScheduler - Scheduler QuartzScheduler_$_NON_CLUSTERED started.
2016-10-25 11:15:01,323 [16] INFO Quartz.Server.QuartzServer - Scheduler started successfully. AutomatiCloudService Version: 3.2.638.2241
2016-10-25 11:15:01,323 [16] INFO Quartz.Server.QuartzServer - Running in time zone: AUS Eastern Summer Time UTC-Offset: 11:00:00
2016-10-25 11:15:01,323 [16] INFO Quartz.Server.QuartzServer - Init License
2016-10-25 11:15:01,324 [16] INFO Quartz.Server.QuartzServer - Licensed for: SURNAME USERNAME@DOMAIN.com.au COMPANYNAME
2016-10-25 11:15:01,325 [16] INFO Quartz.Server.QuartzServer - HWID: HWID
2016-10-25 11:15:30,642 [QuartzScheduler_Worker-1] INFO AWSBackupEC2Job - BackupEC2 job rescheduled
2016-10-25 11:16:33,509 [QuartzScheduler_Worker-2] INFO AWSBackupEC2Job - EBS snapshot created for volume ID: vol-326d95b0 Snapshot ID:snap-c0cacbf3
2016-10-25 11:16:33,535 [QuartzScheduler_Worker-2] INFO AWSBackupEC2Job - CopySnapshot job created for snapshot:snap-c0cacbf3 to region: ap-northeast-1
2016-10-25 11:16:37,128 [QuartzScheduler_Worker-2] INFO AWSBackupEC2Job - EBS snapshot created for volume ID: vol-526d95d0 Snapshot ID:snap-bccb0eb1
2016-10-25 11:16:37,153 [QuartzScheduler_Worker-2] INFO AWSBackupEC2Job - CopySnapshot job created for snapshot:snap-bccb0eb1 to region: ap-northeast-1
2016-10-25 11:16:40,854 [QuartzScheduler_Worker-2] INFO AWSBackupEC2Job - EBS snapshot created for volume ID: vol-d36e9651 Snapshot ID:snap-64bf5556
2016-10-25 11:16:40,877 [QuartzScheduler_Worker-2] INFO AWSBackupEC2Job - CopySnapshot job created for snapshot:snap-64bf5556 to region: ap-northeast-1
2016-10-25 11:16:46,337 [QuartzScheduler_Worker-2] INFO AWSBackupEC2Job - BackupEC2 job rescheduled
2016-10-25 11:17:47,594 [QuartzScheduler_Worker-6] INFO AWSBackupEC2Job - Backup EC2 Instance ID:i-f59167c4 completed.

Is this a bug?

Posted: **Thu Oct 27, 2016 4:08 pm**

Can you please double check that the user you are using in the Europe region has all the required rights.

Stephan

Posted: **Thu Oct 27, 2016 11:30 pm**

Thanks for getting back to me on this Stephan.

I have both my 'default' and 'Ireland' setting using the same Access & Secret Key. This access key maps to a global IAM user with the following permission policy:

Code: Select all

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:*",
                "rds:*"
            ],
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": "IP ADDRESS OF AUTOMATICLOUD HOST"
                }
            },
            "Resource": [
                "*"
            ]
        }
    ]
}

Thus from my understanding I don't think this looks like a permissions problem?

Posted: **Wed Nov 02, 2016 7:14 am**

Note I tested this again after upgrading to the new version 3.3 and it still has the same problem.

Posted: **Thu Nov 03, 2016 7:20 pm**

This seems to be a bug. I will provide a fix in the next days.

Stephan

Posted: **Fri Nov 11, 2016 1:53 pm**

Can you please try 3.3.779.

Stephan

Posted: **Wed Nov 16, 2016 5:15 am**

Hi Stephen,

I just tested against the latest version 3.3.780 and it looks like it's working now. Thanks for fixing!

Regards,
Chris

AutomatiCloud

Backup copy to region failing for 2nd setting

Backup copy to region failing for 2nd setting

Re: Backup copy to region failing for 2nd setting

Re: Backup copy to region failing for 2nd setting

Re: Backup copy to region failing for 2nd setting

Re: Backup copy to region failing for 2nd setting

Re: Backup copy to region failing for 2nd setting

Re: Backup copy to region failing for 2nd setting