Snapshot Management and Restoring from Snapshots¶

Managing Snapshot Job Schedules¶

Job Schedule¶

A job schedule describes an action that OpsCenter will run periodically. Currently, the only type of scheduled job available is a snapshot job.

A job schedule has the following form:

{
    "first_run_date": FIRST_RUN_DATE,
    "first_run_time": FIRST_RUN_TIME,
    "timezone": TIMEZONE,
    "interval": INTERVAL,
    "interval_unit": INTERVAL_UNIT
    "job_params": JOB_PARAMS,
    "id": ID,
    "next_run": NEXT_RUN,
    "last_run": LAST_RUN
}

Data:

FIRST_RUN_DATE (string) -- A date in YYYY-MM-DD format the date for running the job. FIRST_RUN_TIME (string) -- A time in hh:mm:ss format specifying the time to begin running the job. TIMEZONE (string) -- The time zone, listed in the OpsCenter /meta/timezones directory, for the job schedule. For example, GMT, US/Central, US/Pacific, and US/Eastern are valid timezones. INTERVAL (int) -- In conjunction with interval_unit, this controls how often the job is executed. For example, an interval of 2 and an interval_unit of weeks results in a job that will run every two weeks. INTERVAL_UNIT (string) -- The unit of time for interval. Values are minutes, hours, days, or weeks. JOB_PARAMS (dict) -- A dictionary that describes the job. Only the type field is required to be present in the dictionary, and its value is currently limited to backup. Fields for this dictionary are specific to the type; for the backup type, see Backup Job Params. ID (string) -- A unique ID that references a job schedule. Use only for retrieving a job schedule. NEXT_RUN (string) -- The date and time of the next scheduled run. LAST_RUN (string) -- The date and time of the last successful run.

Backup Job Params¶

A JSON dictionary that describes a backup type of Job Schedule. This should be used as the job_params field in the Job Schedule.

{
    "type": "backup",
    "keyspaces": KEYSPACES,
    "cleanup_age": CLEANUP_AGE,
    "cleanup_age_unit": CLEANUP_AGE_UNIT,
    "pre_snapshot_script": PRE_SNAPSHOT_SCRIPT,
    "post_snapshot_script": POST_SNAPSHOT_SCRIPT
}

Data:

KEYSPACES (list) -- A JSON list of keyspace names that should be included in scheduled snapshots. An empty list or null will result in all keyspaces being included in the snapshots. CLEANUP_AGE (int) -- (Optional) In combination with CLEANUP_AGE_UNIT, this specifies the age at which old snapshots should be deleted. A value of 0 will disable automatic snapshot cleanup. This option defaults to a value of 0, meaning automatic snapshot cleanups are disabled by default. CLEANUP_AGE_UNIT (string) -- (Optional) The unit of time for CLEANUP_AGE. Valid values include "minutes", "hours", "days" (the default), and "weeks". PRE_SNAPSHOT_SCRIPT (string) -- (Optional) The file name of a custom script to be automatically run prior to triggering each snapshot. This file must exist within the bin/backup-scripts/ directory where the OpsCenter agent is installed. Only letters, numbers, underscores and hyphens are permitted in the name. POST_SNAPSHOT_SCRIPT (string) -- (Optional) The file name of a custom script to be automatically run after each snapshot is taken. The name of each file included in the snapshot will be passed to the script through stdin. This file must exist within the bin/backup-scripts/ directory where the OpsCenter agent is installed. Only letters, numbers, underscores and hyphens are permitted in the name.

GET /{cluster_id}/job-schedules¶

Retrieve a list of jobs scheduled to run in OpsCenter. Currently the only type of job is a scheduled snapshot.

Path arguments:	cluster_id -- A Cluster Config ID.

Returns a list of Job Schedule objects.

Example:

curl http://127.0.0.1:8888/Test_Cluster/job-schedules

Output:

[
  {
    "first_run_date": "2012-04-19",
    "first_run_time": "18:00:00",
    "id": "19119720-115a-4f2c-862f-e10e1fb90eed",
    "interval": 1,
    "interval_unit": "days",
    "job_params": {
      "cleanup_age": 30,
      "cleanup_age_unit": "days",
      "keyspaces": [],
      "type": "backup"
    },
    "last_run": "2012-04-20 18:00:00 GMT",
    "next_run": "2012-04-21 18:00:00 GMT",
    "timezone": "GMT"
  },
  ...
]

GET /{cluster_id}/job-schedules/{schedule_id}¶

Get the description of a scheduled job.

Path arguments:	cluster_id -- A Cluster Config ID. schedule_id -- A unique ID of the scheduled job that matches the id of a Job Schedule object.

Returns a Job Schedule object.

POST /{cluster_id}/job-schedules¶

Create a new scheduled job. You can create a scheduled job to run one time in the future by specifying an interval of -1 and interval_unit of null.

Path arguments:	cluster_id -- A Cluster Config ID.
Body :	A dictionary in the format of a Job Schedule describing the scheduled job to create. The id, last_run, and next_run fields should be omitted.
Responses:	201 -- Job schedule created successfully

Returns the ID of the newly created job.

Example

curl -X POST
  http://127.0.0.1:8888/Test_Cluster/job-schedules/
  -d
  '{
    "first_run_date": "2012-05-03",
    "first_run_time": "18:00:00",
    "interval": 1,
    "interval_unit": "days",
    "job_params": {
        "cleanup_age": 30,
        "cleanup_age_unit": "days",
        "keyspaces": [],
        "type": "backup"
    },
    "timezone": "GMT"
  }'

Output:

"905391b7-1920-486d-a633-282f22dce604"

PUT /{cluster_id}/job-schedules/{schedule_id}¶

Update a scheduled job.

Path arguments:	cluster_id -- A Cluster Config ID. schedule_id -- A unique ID identifying the schedule job to update.
Body :	A dictionary with fields from Job Schedule that you would like to update.
Responses:	200 -- Schedule updated successfully

Returns null.

Example

curl -X PUT
  http://127.0.0.1:8888/Test_Cluster/job-schedules/905391b7-1920-486d-a633-282f22dce604
  -d
  '{
    "interval": "12",
    "interval_unit": "hours"
  }'

DELETE /{cluster_id}/job-schedules/{schedule_id}¶

Delete a scheduled job.

Path arguments:	cluster_id -- A Cluster Config ID. schedule_id -- A unique ID identifying the schedule job to delete.
Responses:	200 -- Schedule deleted successfully

Returns null.

Example

curl -X DELETE
  http://127.0.0.1:8888/Test_Cluster/job-schedules/905391b7-1920-486d-a633-282f22dce604

Managing Snapshots¶

GET /{cluster_id}/backups¶

Get a list of snapshots that exist in the cluster.

Path arguments:	cluster_id -- A Cluster Config ID.
Opt. params:	match -- A Job Schedule ID to limit snapshots to those created by a particular snapshot schedule.

Returns a dictionary describing snapshots in the cluster. Each key in the dictionary is the ID of the snapshot and each value is a dictionary describing the snapshot.

Example

curl http://127.0.0.1:8888/Test_Cluster/backups

Output:

{
  "opscenter_adhoc_2012-04-19-17-41-12-UTC": {
    "id": "adhoc",
      "keyspaces": {
          "OpsCenter": {
                "cfs": [
                    "events",
                    "events_timeline",
                    "rollups60",
                    ...
                ],
                "nodes": [
                    "127.0.0.1"
                ],
                "size": 23180
            },
            "system": {
                "cfs": [
                    "LocationInfo",
                    "Schema",
                    ...
                ],
                "nodes": [
                    "127.0.0.1"
                ],
                "size": 48961
            },
            "test": {
                "cfs": [
                    "cf1",
                    "cf2"
                ],
                "nodes": [
                    "127.0.0.1"
                ],
                "size": 4543
            }
        },
        "time": 1334857272
    }
}

GET /{cluster_id}/backups/{ks_name}¶

Get a list of snapshots that exist for a keyspace in the cluster.

Path arguments:	cluster_id -- A Cluster Config ID. ks_name -- The keyspace scheduled for snapshot.

Returns data in the same format as GET /{cluster_id}/backups.

POST /{cluster_id}/backups/run¶

Run a one time backup immediately.

Path arguments:	cluster_id -- A Cluster Config ID.
Body :	A JSON dictionary containing the options to use for running this one-time snapshot.

Returns true to indicate a successful snapshot.

Example

Start a snapshot of two keyspaces immediately.

curl -X POST
  http://127.0.0.1:8888/Test_Cluster/backups/run
  -d '{"keyspaces": ["Keyspace1", "Keyspace2"]}'

Output:

The first true response indicates that the snapshot was successful for the specified node.

[
  true,
  [
    null,
    "10.11.12.150"
  ]
],
[
  true,
  [
    null,
    "10.11.12.152"
  ]
]

Note: You can schedule a one-time snapshot in the future using the POST /{cluster_id}/job-schedules. Specify an interval=-1 and interval_unit=null.

Restoring from Snapshots¶

POST /{cluster_id}/backups/restore/{tag}¶

Restore all data from a snapshot.

Path arguments:

cluster_id -- A Cluster Config ID. tag -- The ID of a snapshot to restore. This must match one of the IDs returned by GET /{cluster_id}/backups.

Body :

A JSON dictionary of options, including: throttle: The streaming limit in MB/second. The default behavior is no throttle. excluded-nodes: The list of nodes to exclude from the restore process. force: True or false. True ignores problems, such as some nodes or agents being down, that normally abort restoration. keyspaces: A dictionary mapping keyspace names to a dictionary of options specific to that keyspace. If this parameter is omitted, all keyspaces will be restored. If it is present, only the keyspaces listed in the dictionary will be restored. The set of accepted options per-keyspace include the following: column-families: A list of specific column families to restore. If omitted, all column families will be restored. If omitted, all column families will be restored. truncate: True or false. True first truncates all column families that will be restored. By default, column families are not truncated before restoration.

Returns a Request ID.

Example

SNAPSHOT='opscenter_4a269167-96c1-40c7-84b7-b070c6bcd0cd_2012-06-07-18-00-00-UTC'
curl -X POST
    http://192.168.1.1:8888/Test_Cluster/backups/restore/$SNAPSHOT
    -d '{
      "throttle": 10,
      "excluded-nodes": [
        "192.168.100.7"
      ],
      "keyspaces": {
        "Keyspace1": {
          "column-families: ["users", "dates"],
          "truncate": true
        },
        "OpsCenter": {
          "truncate": false
        }
      },
    }'

Output:

"fee2232c-48ac-11e2-b1b9-e0b9a54a6d93"

POST /{cluster_id}/backups/restore/{tag}/{ksname}¶

Restore a keyspace from a snapshot.

Path arguments:

cluster_id -- A Cluster Config ID. tag -- The ID of a snapshot to restore. This must match one of the ID's returned by GET /{cluster_id}/backups. ksname -- A keyspace to restore.

Body :

A JSON dictionary of options, including: throttle: The streaming limit in MB/second. The default behavior is no throttle. excluded-nodes: The list of nodes to exclude from the restore process. force: True or false. True ignores problems, such as some nodes or agents being down, that normally abort restoration. column-families: A list of specific column families to restore. If omitted, all column families will be restored. truncate: True or false. True first truncates all column families that will be restored. By default, column families are not truncated before restoration.

Returns a Request ID.

Example

SNAPSHOT='opscenter_4a269167-96c1-40c7-84b7-b070c6bcd0cd_2012-06-07-18-00-00-UTC'
curl -X POST
    http://192.168.1.1:8888/Test_Cluster/backups/restore/$SNAPSHOT/Keyspace1
    -d '{
      "throttle": 10,
      "excluded-nodes": [
        "192.168.100.7"
      ],
      "column-families: ["users", "dates"],
      "truncate": true
    }'

Output:

"fee2232c-48ac-11e2-b1b9-e0b9a54a6d93"

POST /{cluster_id}/backups/restore/{tag}/{ksname}/{cfname}¶

Restore a column family from a snapshot.

Path arguments:

cluster_id -- A Cluster Config ID. tag -- The ID of a snapshot to restore. This must match one of the ID's returned by GET /{cluster_id}/backups. ksname -- A keyspace to restore. cfname -- A column family to restore.

Body :

A JSON dictionary of options, including: throttle: The streaming limit in MB/second. The default behavior is no throttle. excluded-nodes: The list of nodes to exclude from the restore process. force: True or false. True ignores problems, such as some nodes or agents being down, that normally abort restoration. truncate: True or false. True first truncates all column families that will be restored. By default, column families are not truncated before restoration.

Returns a Request ID.

Example

SNAPSHOT='opscenter_4a269167-96c1-40c7-84b7-b070c6bcd0cd_2012-06-07-18-00-00-UTC'
curl -X POST
    http://192.168.1.1:8888/Test_Cluster/backups/restore/$SNAPSHOT/Keyspace1/users
    -d '{
      "throttle": 10,
      "excluded-nodes": [
        "192.168.100.7"
      ],
      "truncate": true
    }'

Output:

"fee2232c-48ac-11e2-b1b9-e0b9a54a6d93"