TrueNAS Scale S3 Service to Minio Application Migration

[sretalla]

I just ran up an instance (using a proper certificate for both the API and console endpoints) and it logs in fine.

I note that before I had the right certificate config, the login was not working with the credentials I had set.

 

[kavaa]

I just ran up an instance (using a proper certificate for both the API and console endpoints) and it logs in fine.

I note that before I had the right certificate config, the login was not working with the credentials I had set.

I'm using a Wildcard certificate. Might that be the problem?
And what are you using at:
MinIO Server URL (API)
MinIO Browser Redirect URL

Have mine set to: https://truenas.domain.com both
Cannot leave them blank.

 

[sretalla]

I set mine to https://mi.mydomain.com and https://mic.mydomain.com respectively.

Both domains are correctly forwarded via Nginx Proxy Manager at the respective port (look on your app card) for the API endpoint and the console.

 

[kavaa]

Right, so we cannot use the old S3 location (dataset) or can we? Did anyone found a way to do this safely?
Otherwise we need to create a new location and copy 15TB of Buckets :-( Its split into about 243 buckets...

 

[sretalla]

How to Repatriate From AWS S3 to MinIO

If S3 costs are burning a hole in your pocket, then it's time to start thinking about running MinIO on-premise for your private cloud.

blog.min.io

I guess there may be some clues here on how to transfer data over, but I expect the format is different, so direct re-use wouldn't be an option.

 

[Richard Durso]

Right, so we cannot use the old S3 location (dataset) or can we? Did anyone found a way to do this safely?
Otherwise we need to create a new location and copy 15TB of Buckets :-( Its split into about 243 buckets...

The standalone/file system mode used by TrueNAS S3 Service (release 2021-11-24T23:19:33Z) was dropped from MinIO back in release 2022-10-24T18-35-07Z. Nothing newer than that is going to work. Any path forward needs some kind of export/copy process.

Perhaps you could explore setting up a docker container using a version around 2022-10-24T18-35-07Z, see if it can use the old dataset and then try to determine if that version support exporting data. This Github issue hints that its possible. If that works, you should be able to get a clean export and less of a hack then I documented.

 

[kavaa]

The standalone/file system mode used by TrueNAS S3 Service (release 2021-11-24T23:19:33Z) was dropped from MinIO back in release 2022-10-24T18-35-07Z. Nothing newer than that is going to work. Any path forward needs some kind of export/copy process.

Perhaps you could explore setting up a docker container using a version around 2022-10-24T18-35-07Z, see if it can use the old dataset and then try to determine if that version support exporting data. This Github issue hints that its possible. If that works, you should be able to get a clean export and less of a hack then I documented.

Right, thanks. Still checking which MinIO going to use. What is the best advice? Chart version, Enterprise or regular Docker config? What’s your guys take on this?

Also saw some things to Mirror or replicate. Still got enough space left but its going to be a pain

 

[adam7288]

Any updates on this? Same boat, terabytes of s3 data needing to be migrated but cannot afford downtime.

 

[kavaa]

Any updates on this? Same boat, terabytes of s3 data needing to be migrated but cannot afford downtime.

Not that we know of.
We did the hard thing, create new MinIO via the Apps section and them migrate all the buckets.
We had a bit of downtime during the last sync but we are on the latest versions now.
See my posts in: https://www.truenas.com/community/threads/truenas-scale-minio-and-certificate.111928/#post-774665