I’m aware this general topic has been discussed at length, but I’ve not come across my intended use-case in my research on this forum.
I have a 1TB dataset which is a read-only dataset. (Clarification; it isn’t set as a read-only ZVOL, I’m specifying that the dataset only needs to be read, and will not be written to).
This particular dataset is on a striped mirror of NvME’s.
I have multiple client machines that need to read the data. Sometimes concurrently, but often not concurrently. The new machines I am commissioning have 1.5TB RAM, and ideally the whole dataset will be loaded into RAM on these machines. There will be still a few client machines with 256-512GB RAM where this won’t be possible. These clients will need to read the data as needed. And of course the data will occasionally be re-loaded in the 1.5TB machines, upon reboot.
I’ve found testing over NFS Share hasn’t produced the result I’d like speed-wise, however iSCSI provides enough temporarily, while I sort out an NvME over fabrics solution. At least for now, I iSCSI is the protocol.
I’m trying to avoid having 8 separate iSCSI targets, one for each machine, considering it is the same data.
I’m not opposed to creating a deduplicated dataset with 8 ZVOL’s. However, I’m unsure as to how it will actually function; as I have several deduplicated datasets which serve NFS shares of specific sets of purposely duplicated data, which is done to present the particular sets of data in different categorical/hierarchical ways. But, even in this circumstance for some reason the hit rate isn’t where I’d like it to be. (4tb of unique data is occupying 7tb of space. Which is fine, considering without deduplication it would occupy 20tb. But if the same hit rate is realized on this use-case, it’s several thousands of dollars in additional NvME’s for duplicated data).
So, in conclusion my applied test method has been to have one ZVOL that serves as the target to multiple initiators who are only reading the data. In testing, it works as needed, and I’ve not seen any quantifiable negative impact. However, I’m not an expert, and would like to know what I may be missing; either in potential adverse consequences, or missing in potential solutions/methodologies to execute on the use-case. Perhaps setting-up deduplication in a specific manner which will give me a higher hit-rate, for example. And to answer the potential forthcoming questions of whether or not my use-case necessitates the performance difference between NFS and iSCSI, the answer is yes, it does have a quantifiable impact.
This isn’t a permanent solution. When knowledge, budget, and technology intersect, nvme over fabrics will likely be my semi-permanent solution.
Where are the holes in my execution. What are my alternatives, and/or is there a better methodology?
I have a 1TB dataset which is a read-only dataset. (Clarification; it isn’t set as a read-only ZVOL, I’m specifying that the dataset only needs to be read, and will not be written to).
This particular dataset is on a striped mirror of NvME’s.
I have multiple client machines that need to read the data. Sometimes concurrently, but often not concurrently. The new machines I am commissioning have 1.5TB RAM, and ideally the whole dataset will be loaded into RAM on these machines. There will be still a few client machines with 256-512GB RAM where this won’t be possible. These clients will need to read the data as needed. And of course the data will occasionally be re-loaded in the 1.5TB machines, upon reboot.
I’ve found testing over NFS Share hasn’t produced the result I’d like speed-wise, however iSCSI provides enough temporarily, while I sort out an NvME over fabrics solution. At least for now, I iSCSI is the protocol.
I’m trying to avoid having 8 separate iSCSI targets, one for each machine, considering it is the same data.
I’m not opposed to creating a deduplicated dataset with 8 ZVOL’s. However, I’m unsure as to how it will actually function; as I have several deduplicated datasets which serve NFS shares of specific sets of purposely duplicated data, which is done to present the particular sets of data in different categorical/hierarchical ways. But, even in this circumstance for some reason the hit rate isn’t where I’d like it to be. (4tb of unique data is occupying 7tb of space. Which is fine, considering without deduplication it would occupy 20tb. But if the same hit rate is realized on this use-case, it’s several thousands of dollars in additional NvME’s for duplicated data).
So, in conclusion my applied test method has been to have one ZVOL that serves as the target to multiple initiators who are only reading the data. In testing, it works as needed, and I’ve not seen any quantifiable negative impact. However, I’m not an expert, and would like to know what I may be missing; either in potential adverse consequences, or missing in potential solutions/methodologies to execute on the use-case. Perhaps setting-up deduplication in a specific manner which will give me a higher hit-rate, for example. And to answer the potential forthcoming questions of whether or not my use-case necessitates the performance difference between NFS and iSCSI, the answer is yes, it does have a quantifiable impact.
This isn’t a permanent solution. When knowledge, budget, and technology intersect, nvme over fabrics will likely be my semi-permanent solution.
Where are the holes in my execution. What are my alternatives, and/or is there a better methodology?
