Articles in this section

Setting up cloud storage and configuring jobs

Before you can start creating jobs that send or receive data from the cloud, you need to configure a cloud storage connection in Settings -> Storage Connectors:

Resilio Connect Agents can work with several types of cloud storage: Amazon S3, Azure blob, Azure files, Google cloud and generic S3 Compatible storages. Each type is configured separately. To be able to add cloud storage, the MC license must include a "Cloud" feature in at least one package.  

Once you add your new storage you can use it in as many jobs as you like.

No connection probed

The Management Console itself won't probe the connection to your cloud storage. Connection tests are performed by one of the chosen Agents, or by the configured Agent in the job.

amazon.png   iam.png

  1. Name of the storage (any name that will help you to later understand what this storage is)
  2. Description, optional
  3. Access key ID. Should be copied from your Amazon account
  4. Secret access key. Should be copied from your Amazon account
  5. Region name. Please specify as dash-delimited text as in "Region" column of officially supported AWS regions.
  6. Use bucket. Optional*. Can be left empty, if the access keys have permissions to list the buckets. Otherwise, it won't be possible to test connection to the cloud and browse through buckets.
  7. Starting with MC v3.3, access by Amazon IAM Roles is supported. Bucket checkbox is optional. If it’s checked, the Agents’ connections will be limited to only that bucket. If not - all the buckets that the IAM Roles allow, will be listed by the Agent when configuring the job in the path picker (Permission to list the buckets does not guarantee write access inside the buckets).

Note: the Agent will not report access errors shortly after the role is revoked, until the access token expires or until the Agent is restarted.

(*) Empty bucket name

If you leave it empty, agent will be able to access all buckets by the provided access keys. Thus you MUST specify some bucket name as part of the path during job configuration and that bucket MUST exist in the storage already.

azure2.png

  1. Name of the storage (any name that will help you to later understand what this storage is).
  2. Description, optional
  3. Access key or SAS token. You can get in Home -> Access keys or Shared access signature 
    of your Storage account
  4. Storage account name
  5. Leave endpoint default unless you are using non-default connection endpoint
  6. Use container. Optional*. Can be left empty, if the access keys have permissions to list the containers. Otherwise, it won't be possible to test connection to the cloud and browse through containers.

(*) Empty container name

If you leave it empty, agent will be able to access all containers by the provided access keys. Thus you MUST specify some container name as part of the path during job configuration and that container MUST exist in the storage already.

afiles.png

  1. Name of the storage (any name that will help you to later understand what this storage is).
  2. Description, optional.
  3. Access key or SAS token. You can get in Home -> Access keys or Shared access signature 
    of your Storage account
  4. Storage account name
  5. Leave endpoint default unless you are using non-default connection endpoint
  6. Use share. Optional*. Can be left empty, if the access keys have permissions to list the shares. Otherwise, it won't be possible to test connection to the cloud and browse through buckets.

(*) Empty share name

If you leave it empty, agent will be able to access all shares by the provided access keys. Thus you MUST specify some share name as part of the path during job configuration and that container MUST exist in the storage already.

s3c.png

  1. Name of the storage (any name that will help you to later understand what this storage is).
  2. Description, optional
  3. Your endpoint that agent will attempt to send S3 API requests to*
  4. Access Key ID. Should be provided to you by your S3-compatible storage vendor
  5. Secret access key. Should be provided to you by your S3-compatible storage vendor
  6. Region. Specify bucket's region. 
  7. Checkbox 'Use SSL'. If checked, TLS protocol will be used in communication with the storage.
  8. Use bucket. Optional**.  Can be left empty, if the access keys have permissions to list the buckets. Otherwise, it won't be possible to test connection to the cloud and browse through buckets.

(*) S3 Compatible Storages' peculiarities

1. Wasabi: it's advisable to disable Archive through parameter "Use archive" in the Agent Profile.
2. MatrixStore: it's advisable to place the VM with the Agent as close to the data as possible to avoid connection timeouts.
3. Even though OpenStack and SwiftStack storages support zero-sized objects and symbolic links, these are not synced with the storage by Resilio.
4. Swiftstack: storage Archive is not supported.
5. Backblaze: the Agent ignores etags by default. It's configurable with a custom parameter in the Job Profile.
6. Backblaze: may report error SE_ENC_PEER_VERIFY_ERROR when testing connection to cloud. See here for more details.
7. Storages known to be NOT supported: Amazon Glacier

(**) Empty bucket name

If you leave it empty, agent will be able to access all buckets by the provided access keys. Thus you MUST specify some bucket name as part of the path during job configuration and that bucket MUST exist in the storage already.

google.png

  1. Name of the storage as it appear in MC everywhere
  2. Description, optional
  3. Access key. Get in (or create one) in your project settings
  4. Secret can also be obtained in Storage Settings
  5. Project ID. Can be learned from Project info. 
  6. Use bucket. Optional*. Can be left empty, if the access keys have permissions to list the buckets. Otherwise, it won't be possible to test connection to the cloud and browse through buckets. 

(*) Empty bucket name

If you leave it empty, agent will be able to access all buckets by the provided access keys. Thus you MUST specify some bucket name as part of the path during job configuration and that bucket MUST exist in the storage already.

Create a new application. Be sure to save Client secret value. Application ID and Client secret value will be needed later when configuring connector in the Management Console
https://portal.azure.com/#view/Microsoft_AAD_RegisteredApps/ApplicationsListBlade

Microsoft Graph ‘Application’ level permissions  Files.ReadWrite.All, Sites.Read.All, User.Read should be granted.
Go to ‘API permissions’ section -> Add a permission, in the ‘Microsoft APIs’ tab choose ‘Microsoft Graph’, then ‘Application permissions’, and search for a specific permission. Be sure that “Admin consent” is granted for the added permissions.

On the Management Console go to Storage Connectors menu and add new storage Sharepoint. 

Fill in Tenant ID and Client ID with the information from the Application. Use the saved Client secret value.
Root (site name) is required.

Drive  - Document library name. Optional, if not provided, the Agent will enumerate drives by its permissions. If the drive is not provided, be sure to add it to the path inside the job, or use the folder picker to browse through the drive. 

Be sure to apply "Preset for Sharepoint" in the Agent Profile before configuring the job.
Read here for more details about synchronising with Sharepoint Online

Once you have a storage configured, at least 2 agents to transfer data and 1 agent is ready to take the role ofcloud agent you can create a job with cloud storage.

  1. Apply cloud storage Profile to the cloud agent. Be sure to use "Preset for Sharepoint" for Sharepoint Online synchronization.
  2. Create and configure job normally. Once you decide one of agents should deliver data to a cloud (or get it from cloud), click this agent's path and unwrap the path macro dropdown:
    drop.png
  3. Select "Cloud storage" and either pick pre-configured cloud storage or configure one. Do not use "Direct path" option here!

    drop2.png
  4. Once you picked the storage, enter the path for that storage or browse through the buckets, see below for completing the "Path" field correctly.
    browse.png

(*) Bucket/container/drive name not configured?

- if you left them empty in cloud storage configuration, now you MUST enter it as a first component of the path. It CAN be the only component, or you may add more subfolders
- if you pre-defined them name in an cloud storage configuration, Agent will create the path you enter inside that bucket/container/drive. They must exist on the storage

Cloud Storage path macro is only available for agents of version 2.9 and newer, which have this feature in their license package.

It's not possible to assign cloud storage path to a group, that's not supported. 

Selective Sync is not supported for a cloud storage. If Selective Sync option is checked, path configuration will report an error:

mceclip3.png

Related articles