S3 Transfer Acceleration (TA)

• Amazon S3 Transfer Acceleration makes public Internet transfers to S3 faster, as it leverages Amazon CloudFront’s globally distributed AWS Edge Locations.
• There is no guarantee that you will experience increased transfer speeds. If S3 Transfer Acceleration is not likely to be faster than a regular S3 transfer of the same object to the same destination AWS Region, AWS will not charge for the use of S3 TA for that transfer.
• This is not the best transfer service to use if transfer disruption is not tolerable.
• S3 TA provides the same security benefits as regular transfers to Amazon S3. This service also supports multi-part upload.
• S3 TA vs AWS Snow*
  • The AWS Snow* Migration Services are ideal for moving large batches of data at once. In general, if it will take more than a week to transfer over the Internet, or there are recurring transfer jobs and there is more than 25Mbps of available bandwidth, S3 Transfer Acceleration is a good option.
  • Another option is to use AWS Snowball Edge or Snowmobile to perform initial heavy lift moves and then transfer incremental ongoing changes with S3 Transfer Acceleration.
• S3 TA vs Direct Connect
  • AWS Direct Connect is a good choice for customers who have a private networking requirement or who have access to AWS Direct Connect exchanges. S3 Transfer Acceleration is best for submitting data from distributed client locations over the public Internet, or where variable network conditions make throughput poor.
• S3 TA vs VPN
  • You typically use (IPsec) VPN if you want your resources contained in a private network. VPN tools such as OpenVPN allow you to setup stricter access controls if you have a private S3 bucket. You can complement this further with the increased speeds from S3 TA.
• S3 TA vs Multipart Upload
  • Use multipart upload if you are uploading large files and you want to handle failed uploads gracefully. With multipart upload, each part of your upload is a contiguous portion of the object’s data. You can upload these object parts independently and in any order. If transmission of any part fails, you can retransmit that part without affecting other parts. 
  • For S3 TA, as the name implies, accelerates your transfer speeds, not just for upload but also for download speed. There is no reason why you can’t use S3 TA and multipart upload together, but if you are only handling small files, using multipart upload is not necessary.

AWS Direct Connect

• Using AWS Direct Connect, data that would have previously been transported over the Internet can now be delivered through a private physical network connection between AWS and your datacenter or corporate network. Customers’ traffic will remain in AWS global network backbone, after it enters AWS global network backbone.
• Benefits of Direct Connect vs internet-based connections
  • reduced costs
  • increased bandwidth
  • a more consistent network experience
• Each AWS Direct Connect connection can be configured with one or more virtual interfaces. Virtual interfaces may be configured to access AWS services such as Amazon EC2 and Amazon S3 using public IP space, or resources in a VPC using private IP space.
• You can run IPv4 and IPv6 on the same virtual interface.

• Direct Connect does not support multicast.
• A Direct Connect connection is not redundant. Therefore, a second line needs to be established if redundancy is required. Enable Bidirectional Forwarding Detection (BFD) when configuring your connections to ensure fast detection and failover.
• AWS Direct Connect offers SLA.
• Direct Connect vs IPsec VPN
  • A VPC VPN Connection utilizes IPSec to establish encrypted network connectivity between your intranet and Amazon VPC over the Internet. VPN Connections can be configured in minutes and are a good solution if you have an immediate need, have low to modest bandwidth requirements, and can tolerate the inherent variability in Internet-based connectivity. AWS Direct Connect does not involve the Internet; instead, it uses dedicated, private network connections between your intranet and Amazon VPC.
• You can combine one or more Direct Connect dedicated network connections with the Amazon VPC VPN. This combination provides an IPsec-encrypted private connection that also includes the benefits of Direct Connect.

AWS VPN

• AWS VPN is comprised of two services:
  • AWS Site-to-Site VPN enables you to securely connect your on-premises network or branch office site to your Amazon VPC.
  • AWS Client VPN enables you to securely connect users to AWS or on-premises networks.
• Data transferred between your VPC and datacenter routes over an encrypted VPN connection to help maintain the confidentiality and integrity of data in transit.
• If data that passes through Direct Connect moves in a dedicated private network line, AWS VPN instead encrypts the data before passing it through the Internet.
• VPN connection throughput can depend on multiple factors, such as the capability of your customer gateway, the capacity of your connection, average packet size, the protocol being used, TCP vs. UDP, and the network latency between your customer gateway and the virtual private gateway.
• All the VPN sessions are full-tunnel VPN. (cannot split tunnel)
• AWS Site-to-Site VPN enable you to create failover and CloudHub solutions with AWS Direct Connect.
• AWS Client VPN is designed to connect devices to your applications. It allows you to choose from OpenVPN-based client.

Snowball Edge

• Snowball Edge is a petabyte-scale data transport solution that uses secure appliances to transfer large amounts of data into and out of AWS.
• Benefits of Snowball Edge include:
  • lower network costs,
  • Shorter transfer times,
  • and security using 256-bit encryption keys you manage through AWS Key Management Service (KMS)..
• Options for device configurations
  • Storage optimized – this option has the most storage capacity at up to 80 TB of usable storage space, 24 vCPUs, and 32 GiB of memory for compute functionality. You can transfer up to 100 TB with a single Snowball Edge Storage Optimized device.
  • Compute optimized – this option has the most compute functionality with 52 vCPUs, 208 GiB of memory, and 7.68 TB of dedicated NVMe SSD storage for instance. This option also comes with 42 TB of additional storage space.
  • Compute Optimized with GPU – identical to the compute-optimized option, save for an installed GPU, equivalent to the one available in the P3 Amazon EC2 instance type.
• Similar to Direct Connect, AWS Snowball Edge is physical hardware. It includes a 10GBaseT network connection. You can order a device with either 50TB or an 80TB storage capacity.
• Data transported via Snowball Edge are stored in Amazon S3 once the device arrives at AWS centers.
• AWS Snowball Edge is not only for shipping data into AWS, but also out of AWS.
• AWS Snowball Edge can be used as a quick order for additional temporary petabyte storage.
• You can cluster Snowball Edge devices for local storage and compute jobs to achieve 99.999 percent data durability across 5–10 devices, and to locally grow and shrink storage on demand.
• For security purposes, data transfers must be completed within 360 days of a Snowball Edge’s preparation.
• When the transfer is complete and the device is ready to be returned, the E Ink shipping label will automatically update to indicate the correct AWS facility to ship to, and you can track the job status by using Amazon Simple Notification Service (SNS), text messages, or directly in the console.
• Snowball Edge is the best choice if you need to more securely and quickly transfer terabytes to many petabytes of data to AWS. Snowball Edge can also be the right choice if you don’t want to make expensive upgrades to your network infrastructure, if you frequently experience large backlogs of data, if you’re located in a physically isolated environment, or if you’re in an area where high-bandwidth Internet connections are not available or cost-prohibitive.
• For latency-sensitive applications such as machine learning, you can deploy a performance-optimized SSD volume (sbp1). Performance optimized volumes on the Snowball Edge Compute Optimized device use NVMe SSD, and on the Snowball Edge Storage Optimized device they use SATA SSD. Alternatively, you can use capacity-optimized HDD volumes (sbg1) on any Snowball Edge.
• If you will be transferring data to AWS on an ongoing basis, it is better to use AWS Direct Connect.
• If multiple users located in different locations are interacting with S3 continuously, it is better to use S3 TA.

• You cannot export data directly from S3 Glacier. It should be first restored to S3.

Snowmobile

• Snowmobile is Snowball Edge with larger storage capacity. Snowmobile is literally a mobile truck.
• Snowmobile is an Exabyte-scale data transfer service.
• You can transfer up to 100PB per Snowmobile.
• Snowmobile uses multiple layers of security to help protect your data including dedicated security personnel, GPS tracking, alarm monitoring, 24/7 video surveillance, and an optional escort security vehicle while in transit. All data is encrypted with 256-bit encryption keys you manage through the AWS Key Management Service (KMS).
• After the data transfer is complete, the Snowmobile will be returned to your designated AWS region where your data will be uploaded into the AWS storage services such as S3 or Glacier.
• Snowball Edge vs Snowmobile
  • To migrate large datasets of 10PB or more in a single location, you should use Snowmobile. For datasets less than 10PB or distributed in multiple locations, you should use Snowball Edge.
  • If you have a high speed backbone with hundreds of Gb/s of spare throughput, then you can use Snowmobile to migrate the large datasets all at once. If you have limited bandwidth on your backbone, you should consider using multiple Snowball Edge to migrate the data incrementally.
  • Snowmobile does not support data export. Use Snowball Edge for this cause.
• When the data import has been processed and verified, AWS performs a software erasure based on NIST guidelines.

References:

https://aws.amazon.com/cloud-data-migration/
https://aws.amazon.com/s3/faqs/#s3ta
https://aws.amazon.com/directconnect/faqs/
https://aws.amazon.com/vpn/faqs/
https://aws.amazon.com/snowball/faqs/
https://aws.amazon.com/snowmobile/faqs/