Slow upload speeds using S3 cloud-sync

[bew]

My total upload speed to my Storj bucket seems to be limited at ~2.3MiB/s. (running 2 separate upload tasks)

My network's max upload speed is capped at around ~18MiB/s so I'm only using a fraction of it.

I'm running TrueNAS 13 U3, my sync task is set-up to push & sync the data

Does anyone know what my bottleneck is? My CPU & memory usage are very low

 

[gwsteffens]

I'm seeing the same upload speed of about 2.2 MiB/s. I also have a max upload speed of about 18MiB/s. I'm running TrueNas Core 13.0-U3.1.
I also have no idea what the bottleneck may be. Could it be limited by the Storj servers?

 

[jcassette]

Same here. I guess it's because of S3. With uplink I get ~30MiB/s.

 

[Jamberry]

Doesn't the TrueNAS "cloud sync" use the native integration instead of S3?
S3 is trough their centralised S3 gateways, which seem to be not great when it comes to performance.

 

[jcassette]

I guess it's S3 since TrueNAS needs S3 credentials: https://docs.storj.io/dcs/how-tos/ix-systems-truenas-integration#_RpGo
Native clients and rclone integration need an Access Grant instead: https://docs.storj.io/dcs/how-tos/sync-files-with-rclone/rclone-with-native-integration
Maybe this could help: https://docs.storj.io/dcs/how-tos/how-to-hod-rod-file-transfer-performance-on-storj-dcs

 

[kcurt]

I'm seeing the same upload speed of about 2.2 MiB/s. I also have a max upload speed of about 18MiB/s. I'm running TrueNas Core 13.0-U3.1.
I also have no idea what the bottleneck may be. Could it be limited by the Storj servers?

 

[kcurt]

I'll guess here and say your max upload is 18Mib/s (small b or bits), So if we divide that by 8 bits/byte, you get 2.25MiB/s. SO I believe you actually are maxing out. Hope this is of some help.

 

[gwsteffens]

Hi Kcurt, You are absolutely right! I usually catch this sort of thing but it escaped my notice this time.
Thanks for pointing this out!