NTFS disk import - no progress?

[gadgetbazza]

My role as a moderator has little to do with this.

I would have thought that you should be mindful of all of your posts since you will be setting an example?

Unlike that QNAP or Synology product that you paid hundreds or even thousands of dollars for, TrueNAS is giving you a product free to use, and I simply think your attitude seems a bit entitled and offensive.

Wow, well then you must have mastered the ability to determine ones attitude purely from written text, a mistake that many have made unfortunately. Try re-reading it without any red mist in front of you and you can see that this is in fact a perfectly valid discussion. You have the cheek to call me entitled and offensive because I am questioning the behaviour of a piece of functionality? Strangely only you seem to be having this "attitude", maybe check the mirror once in a while?

There are a lot of times in computer science when "one-shot use" features are not outfitted with maximal bells and whistles.

Finally a fair comment. I'll accept that this is probably a one-shot use feature, but it's one of the first ones that I personally would expect to use when setting up a NAS. However, all that was needed was to acknowledge that this wasn't possible in the product currently or that it didn't provide the intuitive feedback that I was looking for and that it may be a useful enhancement that may or may not ever be deemed a priority?

Maybe this is a mis-understanding on the positioning of this product for me. On the one hand you have a free open source product aimed at a free community which has many apps on the home page suggesting it will be used for domestic purposes such as Plex and therefore you would have to expect that this would be outfitted with an easy user experience. On the other hand, you have people representing it as a tool for large commercial NAS solutions (which would be paid for) as that is their focus. I'm sure it's technically capable of both, but my post reflects the experience of a user closer to the domestic end of that spectrum and the hiccups experienced.

Please don't keep repeating that I haven't paid for it, I'm still contributing right here and there are thousands of other users out there using it for free, at least I'm providing some feedback. Just because I haven't contributed code to this project, it doesn't mean I don't play my part in other communities or even pay / sponsor some of the other products I use (by choice), nor that I'm not allowed to voice an opinion (unless of course I'm not "entitled" enough????)

 

[Davvo]

I'll accept that this is probably a one-shot use feature, but it's one of the first ones that I personally would expect to use when setting up a NAS.

Consider iX Systems' target being companies with production-level requirements, do they need this?

That being said, I too agree that if something is in the menu I expect it to work: it's the same line of thought of plugins.
However since I'm not paying I can't complain too much.

 

[mistermanko]

Just as a conceptional question and having never tried it myself, wouldn't firing up a Windows VM and passthrough the NTFS disk to the VM be feasible? Either USB passthrough with a SATA to USB thingy attached or HBA passthrough?
Sounds managable, right?

 

[gadgetbazza]

Consider iX Systems' target being companies with production-level requirements, do they need this?

That being said, I too agree that if something is in the menu I expect it to work: it's the same line of thought of plugins.
However since I'm not paying I can't complain too much.

That's probably fair, I can't see a commercial requirement for this, as they will have large amounts of data stored elsewhere and not in a portable fashion to attach locally as I have done so will be forced to transfer over the wire. Hence my question about whether I am viewing this from the more domestic end of the positioning.

Yes I'm not paying (at least until I'm at a place where I feel that I would like to donate if that's possible), but I wasn't intending this to be a "complaint"?!

 

[gadgetbazza]

Just as a conceptional question and having never tried it myself, wouldn't firing up a Windows VM and passthrough the NTFS disk to the VM be feasible? Either USB passthrough with a SATA to USB thingy attached or HBA passthrough?
Sounds managable, right?

Feasible for sure, in fact, probably wouldn't have needed the Windows VM as I'm sure I could have got a Mac or Linux host to read the disk, but as I mentioned, the NTFS access wasn't the real issue, it's the experience.

My original reason for attaching the disk direct was a) the content was already on the drive and b) transfer speed would have been significantly longer over the wire I believe?

 

[Ericloewe]

Maybe this is a mis-understanding on the positioning of this product for me. On the one hand you have a free open source product aimed at a free community which has many apps on the home page suggesting it will be used for domestic purposes such as Plex and therefore you would have to expect that this would be outfitted with an easy user experience. On the other hand, you have people representing it as a tool for large commercial NAS solutions (which would be paid for) as that is their focus. I'm sure it's technically capable of both, but my post reflects the experience of a user closer to the domestic end of that spectrum and the hiccups experienced.

Yeah, definitely the case, to some extent at least.

Thing is, TrueNAS Core (and Scale, after it matures) are lightly cut-down versions of TrueNAS Enterprise, which iX ships on their own hardware exclusively and this is aimed squarely at the enterprise market.

The mix of people using the free versions is an odd one, from hobbyists with trivial needs to companies who figure they can run without getting iX involved (which is fair, too). But the development target is mainly the one paying the bills, which is enterprise.
FreeNAS and TrueNAS have also long suffered from terrible expectations management mostly induced by:

1. FreeNAS 0.7, that evolved into the current XigmaNAS and has nothing reallly to do with FreeNAS 8.x+ and TrueNAS, which was a decidedly jankier product aimed at the hobbyist community.
2. Terrible, terrible articles, videos and similar pieces by a variety of services, misleading users into thinking that TrueNAS is something to throw on an old machine to get a bit of utility out of it for a while longer, or worse. These are fortunately few and far between these days, but they were everywhere eight years ago and a constant irritation.
3. Free and reasonably easy to use tend to paint an incomplete picture of the level of knowledge that tends to allow for a smooth experience (it should be noted that TrueNAS Enterprise includes support contracts, so customers are fairly hands-off for tricky operations or major design decisions).
4. The apps or whatever anyone wants to call them are very visible and appeal strongly to home users, as you says. This was long a feature exclusive to FreeNAS, not present in what is now TrueNAS Enterprise - no doubt, some of the reasoning was indeed to target home users.

 

[Ericloewe]

Just as a conceptional question and having never tried it myself, wouldn't firing up a Windows VM and passthrough the NTFS disk to the VM be feasible? Either USB passthrough with a SATA to USB thingy attached or HBA passthrough?
Sounds managable, right?

Sure, but if adding a transfer status widget hasn't been an engineering priority, the VM option sounds even worse to me.

 

[Ericloewe]

b) transfer speed would have been significantly longer over the wire I believe?

Well, it depends. If the disk can stream the data with few seeks, probably. If it's bound by random I/O, it might actually be faster with Windows dealing with NTFS.

 

[jgreco]

I would have thought that you should be mindful of all of your posts since you will be setting an example?

That is not the deal. A bunch of us were conscripted into moderatorship back in the day. At no point have I given up my rights to hold an opinion as I am not a paid iXsystems employee but rather just another community member. The fact that you are hearing something that you don't care to hear could be something for you to reflect upon.

Put differently, I will not allow you to use my moderator status to influence me into silence when I say something reasonable that you don't care to hear.

 

[Davvo]

A bunch of us were conscripted into moderatorship back in the day.

 

[ChrisRJ]

Wow, well then you must have mastered the ability to determine ones attitude purely from written text, a mistake that many have made unfortunately.

In the absence of being able to read someone else's mind or have a direct conversation, that is indeed the next best thing one can do.

While I had finally decided against doing so, I was also tempted at the time to write something about your wording. I work in mission-critical software, mostly as a developer, but also on the business side to an extent. So I will concede that the positioning of TrueNAS is a bit unusual compared to everything else I am aware of. But I also think that it is a creative approach, in that it creates awareness beyond what conventional marketing could achieve, unless we throw insane amounts of money at it.

What caused irritation for me was when you basically said "this is opensource, plus they advertise it in a certain way, therefore I am part of the target audience and can demand features behave in a specific way". That is how I perceived things. If you tell me that is not what you meant to say, that is absolutely fine. It would also underline how easily purely written communication can create misunderstandings.

 

[anodos]

Sure, but if adding a transfer status widget hasn't been an engineering priority, the VM option sounds even worse to me.

I glanced more closely at the code for the NTFS import status indicator and saw a bug. Should be fixed in 22.12.1. I'm still of the opinion though that this is the wrong way to go about data ingest. :)

 

[gadgetbazza]

That is not the deal. A bunch of us were conscripted into moderatorship back in the day. At no point have I given up my rights to hold an opinion as I am not a paid iXsystems employee but rather just another community member. The fact that you are hearing something that you don't care to hear could be something for you to reflect upon.

Put differently, I will not allow you to use my moderator status to influence me into silence when I say something reasonable that you don't care to hear.

I guess you accepted it though? I wouldn't suggest you don't have an opinion, but manners on the other hand...

I never tried to silence you, if you think calling someone selfish and rude is reasonable, then we have very different opinions of manners clearly.

 

[gadgetbazza]

In the absence of being able to read someone else's mind or have a direct conversation, that is indeed the next best thing one can do.

That's the point though, it's your interpretation, not necessary the one of the author, and one could argue that either or both parties have mis-interpreted it, hence why you should always try to review written communication from multiple view points and not assume that people are being nasty. Responding in an aggressive manner is the only "rude" posting that I see on here.

What caused irritation for me was when you basically said "this is opensource, plus they advertise it in a certain way, therefore I am part of the target audience and can demand features behave in a specific way". That is how I perceived things. If you tell me that is not what you meant to say, that is absolutely fine. It would also underline how easily purely written communication can create misunderstandings.

See, you read something that I didn't write... Where did I "demand" anything?

 

[gadgetbazza]

Well, it depends. If the disk can stream the data with few seeks, probably. If it's bound by random I/O, it might actually be faster with Windows dealing with NTFS.

So I moved 10Tb of data in just under 12 hours over USB host. What would the estimate be using a 1Gbps port with a Windows machine? My guess would be somewhere around 36 hours?

 

[gadgetbazza]

I glanced more closely at the code for the NTFS import status indicator and saw a bug. Should be fixed in 22.12.1. I'm still of the opinion though that this is the wrong way to go about data ingest. :)

Great, I'm glad something good has come out of this.

And having read the constructive posts, I too agree that for a corporate deployment, you wouldn't ingest data the way that I have. But as a user coming to the party with content on a USB drive I thought it a reasonable way to achieve it, given that the feature is there.

Thanks

 

[Davvo]

So I moved 10Tb of data in just under 12 hours over USB host. What would the estimate be using a 1Gbps port with a Windows machine? My guess would be somewhere around 36 hours?

With a 10 Gbps network it would have been way faster.

But as a user coming to the party with content on a USB drive I thought it a reasonable way to achieve it, given that the feature is there.

I agree.

 

[Ericloewe]

So I moved 10Tb of data in just under 12 hours over USB host. What would the estimate be using a 1Gbps port with a Windows machine? My guess would be somewhere around 36 hours?

30ish hours. Transferring over 1GbE is likely to be slower outside of extreme edge cases with extreme fragmentation and such.

 

[Whattteva]

So I moved 10Tb of data in just under 12 hours over USB host. What would the estimate be using a 1Gbps port with a Windows machine? My guess would be somewhere around 36 hours?

Must be a really high quality external. I've never been able to get above 50 MB/s even on a USB 3.0 external HDD, so 1G ethernet + internal SATA drives are still over twice as fast for all my USB externals.

 

[jgreco]

I guess you accepted it though? I wouldn't suggest you don't have an opinion, but manners on the other hand...

No, I woke up one morning to find myself conscripted into moderatorship. No acceptance of an offer was involved.

As for manners, that's a matter of opinion. I don't think your manners here are anything to write home about. You do not appear to appreciate the generous gift that iXsystems has provided to you. As @ChrisRJ says, perhaps you didn't mean it like this, but at least to me, your attitude comes across poorly.