Amazon S3 Plugin for testing

Status
Not open for further replies.

mute

Dabbler
Joined
Dec 8, 2013
Messages
19
Do you have any more details about what this plugin is capable of? I recently found myself needing s3cmd on one of our TrueNAS systems, so I installed it, and compiled gpg on a FreeBSD 9.2 system so I could use the -e option for client side encryption.

Having gpg built into the s3cmd plugin would make it perfect for our needs and preclude the need for me to maintain gpg and the associated libraries. :)
 

matto

FreeBSD Advocate
iXsystems
Joined
Sep 4, 2012
Messages
46
Hi mute,

We will go ahead and get it rebuilt with gpg ;)

I'll post it back up here tomorrow for testing. We just added it to the FreeNAS repo but that's without gpg. Also, send me an email to matt at ixsystems dot com and we can discuss getting it in your next TrueNAS upgrade :D
 

parkedin

Cadet
Joined
Aug 4, 2014
Messages
1
Thanks for creating this plugin.

I'm newer to FreeNAS and have been playing with a few configurations. I have an Amazon S3 account and wanted to leverage my Free Tier and installed this plugin to try and learn a bit. Unfortunately, I've been mostly unsuccessful in using it, though, I have been able to get a file uploaded so I know the installation and configuration are ok.

In the interest of constructive feedback, here's what I have seen.

I installed and added storage via the GUI but I didn't quite understand the source and destination options. I initially thought the source and destination was like the Add Storage dialog with source and destination mount points within the jail. Oops. Good thing it wasn't working for me. haha.

I tried again, using the source watch dir within my jail but could understand the destination. I tried various iterations of my bucket/folder (s3://bucket/folder, /bucket/folder, /differentbucket/folder, etc.).

From my perspective, the service was on but not working. I'm a FreeNAS rube so perhaps I should have been watching a particular logdir other than messages? It also wasn't running as a service to verify status.

I ran s3cmd --help and then googled up http://s3tools.org/usage and dropped to the shell.

I was able to write a text file and then send it to my s3://bucket/folder. When I tried an mp3, it failed with:

{{{
[root@s3cmd_1 ~]# s3cmd put ~/testfile.1 s3://bucket/folder/
/root/testfile.1 -> s3://bucket/folder/testfile.1 [1 of 1]
20 of 20 100% in 0s 101.72 B/s done
WARNING: Redirected to: bucket.region.amazonaws.com
/root/testfile.1 -> s3://bucket/folder/testfile.1 [1 of 1]
20 of 20 100% in 0s 32.63 B/s done
}}}

{{{
# s3cmd put ~/FloatOn s3://bucket/folder/
/root/FloatOn -> s3://bucket/folder/FloatOn [1 of 1]
1736704 of 6477474 26% in 1s 1418.16 kB/s failed
WARNING: Upload failed: /aws3/FloatOn ([Errno 32] Broken pipe)
WARNING: Retrying on lower speed (throttle=0.00)
WARNING: Waiting 3 sec...
}}}

This repeated over and over. Initially I suspected that it may have been because of my jail mount so I copied the file into my home within the jail, stripped spaces, then file extension.

I haven't collected a packet trace, but I can if it would be helpful.

My takeaway suggestions would be:
  1. Update OP with link to CLI usage.
  2. Plugin settings menu needs context and example usage. Do I use s3:// prefix, /, bucket/folder, etc?
  3. Any additional information, anywhere, will be helpful. I didn't see it running in top but I saw Python. Is it possible to add it as a service to verify whether it's running?
  4. The CLI tools were able to help me determine the configuration was good and even put a file. If there's a log, perhaps a path to it in the GUI settings would help. I just kept refreshing my AWS interface and watched iftop for progress.
Thanks.
 

emk2203

Guru
Joined
Nov 11, 2012
Messages
573
Any news on the how-to?
 

ndboost

Explorer
Joined
Mar 17, 2013
Messages
78

dpearcefl

Contributor
Joined
Aug 4, 2015
Messages
145
ndboost, can you please repost a working URL for us?
 

heimos

Dabbler
Joined
Nov 9, 2016
Messages
29
bump
 
Status
Not open for further replies.
Top