Note that this step configures a storage pool as the disk or disks on which DPM stores replicas and recovery points for protected data. This pool is part of the DPM configuration and is separate from the Storage Spaces pool used to create the data volumes described in the previous section. For more information on DPM storage pools see Configure disk storage and storage pools.
Dedup requires a special set of configuration options to support virtualized DPM storage due to the scale of data and size of individual files. These options are global to the cluster or the cluster node. Dedup must be enabled and the cluster settings must be individually configured on each node of the cluster.
To do this run the following PowerShell command on each node of the cluster:. Tune dedup processing for backup data files -Run the following PowerShell command to set to start optimization without delay and not to optimize partial file writes. Note that by default Garbage Collection GC jobs are scheduled every week, and every fourth week the GC job runs in "deep GC" mode for a more exhaustive and time intensive search for data to remove.
For the DPM workload, this "deep GC" mode does not result in any appreciative gains and reduces the amount of time in which dedup can optimize data. We therefore disable this deep mode. Tune performance for large scale operations -Run the following PowerShell script to:. HashIndexFullKeyReservationPercent: This value controls how much of the optimization job memory is used for existing chunk hashes, versus new chunk hashes.
EnablePriorityOptimization: With files approaching 1TB, fragmentation of a single file can accumulate enough fragments to approach the per file limit. Optimization processing consolidates these fragments and prevents this limit from being reached.
By setting this registry key, dedup will add an additional process to deal with highly fragmented deduped files with high priority.
If they were to run at the same time, additional overhead to switch between the operations could be costly and result in less data being backed up or deduplicated on a daily basis. We recommended you configure dedicated and separate deduplication and backup windows. The recommended guidelines for scheduling are:. Set up weekend dedup schedules separately, using that time for garbage collection and scrubbing jobs.
Deduplication is scheduled for the remaining 16 hours of the day. Note that the actual dedup time you configure will depend on the volume size. See Sizing Volumes for Data Deduplication for more information. A hour deduplication window starting at 6 AM after the backup window ends would be configured as follows from any individual cluster node:.
Whenever the backup window is modified it's vital that the deduplication window is modified along with it so they don't overlap. The deduplication and backup window don't have to fill up the full 24 hours of the day, but it's highly recommended that they do to allow for variations in processing time due to expected daily changed in workloads and data churn.
After a set of files has been deduplicated there can be a slight performance cost when accessing the files. This storage can be used by DPM directly for short term retention. If you a large enough size to warrant a storage array, regardless of de-dupe support, I would recommend looking at using thin provisioning to save space. A number of DPM users have contacted me either directly or indirectly expressing interest in our DPM deduplication, compression and encryption technology.
We are typically able to help them. It would be of interest to me to establish a line of communication with my product and product marketing counterparts at Microsoft and was hoping you might be able to give me some assistance and direction in that. Exar is an OEM company and will sell bitwacker cards of quantity or more. BridgeSTOR is the marketing and sales for the card and software.
I am happy to help end users in their quest for deduplication for DPM! See www. Also it is safe to assume that DPM will provide no native dedup algorithms? The problem we had have is that when DPM backs up to Tape VTL it creates a snapshot on the required volumes locally , mounts it, and does a full write to Tape.
When you have got several file servers with millions of files copying to tape consumes a lot of ressources of the DPM Servers OS.
But probably we are going to purchase some CS because the dedup ration is quite impressive with the new firmware and inline dedup!! We had a bit of trouble configuring and getting it going and there are a few caveats we did encounter However as Mike pointed out before about using a VTL they seems to be alot more functional. As you can see we didnt write to tape at all. Any information on this will be appreciated.
We were using 2 solution due to a company athat was aquired, both perform exactly the same. Quantum is cheaper, EMC is alot more scalable and easier to setup. I will be posting another blog with my findings. For now you can visit www. You must be logged in to post a comment. This means that workloads that have idle time, such as in the evening or on weekends, are excellent candidates for deduplication, and workloads that run all day, every day may not be.
Workloads that have no idle time may still be good candidates for deduplication if the workload does not have high resource requirements on the server. Before enabling Data Deduplication, you must choose the Usage Type that most closely resembles your workload. There are three Usage Types included with Data Deduplication. You can find more information on excluding file extensions or folders and selecting the deduplication schedule, including why you would want to do this, in Configuring Data Deduplication.
If you are running a recommended workload, you're done. For other workloads, see Other considerations. This is particularly useful for running the Data Deduplication PowerShell cmdlets remotely against a Nano Server instance. I want to run Data Deduplication on the dataset for X workload. Is this supported? Aside from workloads that are known not to interoperate with Data Deduplication , we fully support the data integrity of Data Deduplication with any workload.
Recommended workloads are supported by Microsoft for performance as well. The performance of other workloads depends greatly on what they are doing on your server. You must determine what performance impacts Data Deduplication has on your workload, and if this is acceptable for this workload. What are the volume sizing requirements for deduplicated volumes? In Windows Server and Windows Server R2, volumes had to be carefully sized to ensure that Data Deduplication could keep up with the churn on the volume.
This typically meant that the average maximum size of a deduplicated volume for a high-churn workload was TB, and the absolute maximum recommended size was 10 TB.
0コメント