Backing Up Aggregate

Recovering from Briefcase

Use ODK Briefcase to back up all forms and submissions on Aggregate. You can pull forms from your Aggregate server into your local machine using Briefcase. In particular, Briefcase's command line interface makes this easier.

Recovering database from MySQL dump

  1. Stop running Tomcat.
  2. Upgrade to the latest version of Aggregate.
  3. Finally, restore it from MySQL dump. An SQL dump of a database is a common method to safely store away a snapshot of the database for archival purposes or to migrate data between database instances, e.g. between two major system releases. The content of a SQL dump is a large collection of SQL commands in ASCII. Running the script will recreate the database in the same state as it was when the dump was created. The primary tool to consider for making an ASCII dump is mysqldump, which includes a wide variety of options.
$ mysqldump [ options ] [ dbname ]

Some of the useful options are:

  • -h hostname or --host=hostname specifies host to connect to.
  • -p portnr or --port=portnr specifies port to connect to.
  • -u user or --user=user specifies user id.
  • -d database or --database=database specifies database to connect to.

To take a backup of database:

$ mysqldump database > backup-file.sql;

To restore a database:

$ mysql database < backup-file.sql;

To copy a database from one server to another

$ mysqldump --opt database | mysql --host=remote_host -C database

remote_host indicates a remote server where you want to take backup.

Note

Creation of the dump respects your credentials, which means you only can dump the tables you have access to.

Backup and recovery on Google App Engine

Create backup

  1. Open a browser to Google Cloud Platform and click on Console in the top right corner.
Image showing console option.
  1. Sign in with a Gmail account which you used for Aggregate installation.
Image showing sign in window.
  1. Choose the project id for your ODK Aggregate server by clicking on the project dropdown in the top left corner.
Image showing project dropdown. Image showing project selection box.
  1. Click on the menu icon (three horizontal bars) to the left of Google Cloud Platform in the upper left side of the screen and then select Datastore from the menu. Click on Admin in the dropdown.
Image showing Datastore and Admin option.
  1. Enable Cloud Datastore Admin access by clicking on Enable Datastore Admin.
Image showing Enable Datastore Admin option.
  1. Then, click on Open Datastore Admin.
Image showing Open Datastore Admin option.

Tip

It is recommended to disable writes during creation of backup. To disable writes, click on Disable writes on the Admin page.

Image showing Disable writes option.

Warning

Your Aggregate server may become unstable when you disable writes. It will be fine when you enable writes again.

  1. Select the entity kinds that you wish to back up and then click on Backup Entities.
Image showing backup entities selection.
  1. A backup form will be displayed.
Image showing backup form.

Note

  • A backup name is supplied and it includes a datestamp. You must change this value if you make more than one backup per day because a backup is not made if a backup of the same name already exists.
  • The default queue is used for the backup job; you can use this in most cases. If you use a non-default queue for backup/restore, you can only specify the target ah-builtin-python-bundle in queue.yaml. You cannot use any other targets. To know more about queues, see this.
  • Select Google Cloud Storage as the backup storage location.

In the bucket name box, enter your-project-id.appspot.com. You can alternatively preface the bucket name with /gs/, for example, /gs/[BUCKET_NAME].

Note

Buckets are containers where your backup will be stored. You can also create buckets for your project.

Now click on Backup Entities to start the backup jobs.

  1. A job status page is displayed. Click on Back to Datastore Admin to see the backup status.
Image showing backup job status page.

You can abort a backup by selecting a backup from the list of pending backups and clicking on Abort.

Image showing Abort and Info option.

Warning

When you abort a backup job, App Engine attempts to delete backup data that has been saved up to that point. However, in some cases, some files can remain after the abort. You can locate these files in the location you chose for your backups in Google Cloud Storage and safely delete them after the abort completes. The names of such files start with the following pattern: datastore_backup_[BUCKET_NAME].

Click on Info to get more information about the backup. On the info page, click Back to Datastore Admin to return to the main Cloud Datastore Admin screen.

Image showing backup info.

Tip

After the backup is complete, if you disabled Cloud Datastore writes, re-enable them by going to Admin page and clicking on Enable writes.

Image showing Enable writes option.

Restoring data from Backup

  1. Go to the Admin page as described in the creation of backup.
Image showing Datastore and Admin option.

Tip

Disable Cloud Datastore writes for your application. It's normally a good idea to do this to avoid conflicts between the restore and any new data written to Cloud Datastore. To disable writes, click on Disable writes on the Admin page.

Image showing Disable writes option.
  1. Click on Open Datastore Admin.
Image showing Open Datastore Admin option.
  1. In the list of available backups, select the backup that you want to restore from and click on Restore. You can click on Info to get more information about the backup. To delete a backup, select the backup and click on Delete.
Image showing Restore, Info and Delete options.
  1. In the advisory page that is displayed, notice the list of entities with checkboxes. By default, all of the entities will be restored. Uncheck the checkbox next to each entity that you don't want to restore. Click on Restore at the bottom of the page to start the restoration.
Image showing restore option.

Note

In the advisory page, notice that the default queue, with its pre-configured performance settings, is used for the restore job. Change this to another queue that you have configured differently if you need different queue performance characteristics, making sure the queue chosen does not have any target specified in queue.yaml other than ah-builtin-python-bundle. To know more about queues, see this.

  1. A job status page is displayed. Click on Back to Datastore Admin to see the status of the restore.
Image showing job status page. Image showing restore status.

Tip

After the restore is complete, if you disabled Cloud Datastore writes, re-enable them by going to Admin page and clicking on Enable writes.

Image showing Enable writes option.

Note

  • If you back up your data using Google Cloud Storage, you can restore backups to applications other than the application used to create the backup. To restore backup data from a source application to a target application, see this guide.
  • Google has new beta service for exporting and importing. Only Cloud Platform projects with billable accounts can use the export and import functionality.