Huawei Cloud recently released new types of Huawei ECS I believe that we all have many questions, especially about the advantages and disadvantages of the new Huawei cloud storage cloud The server What are the advantages and disadvantages of? In order to meet the diversified business needs of customers, today we will focus on the local storage type of Huawei ECS designed for big data scenarios, which aims to solve the problem of massive data processing services on the cloud and fully meet the requirements of Hadoop big data services Huawei Business requirements in terms of ECS storage performance, capacity, stability, etc. Next, let's take a look with the Micronet:

01 Applicable scenarios

Local storage type Huawei ECS is suitable for users who need big data analysis and massive storage, especially for distributed file storage (HDFS), parallel computing (MapReduce) and other related businesses. Huawei The data disk of ECS adopts the design of local storage. Currently, 1-20 8TB SATA disks are provided for mounting to ensure the high requirements of customer business on storage bandwidth and capacity at a lower cost.

02 Product advantages

• The size of a single data disk is up to 8TB. A single instance can provide up to 20 data disks [1] and up to 160TB of free space, providing massive storage resources for big data services.

• The sequential read/write performance of a single disk is up to 250MB/s [2], and the maximum throughput of the instance is close to 5GB/s, reducing the read and write time of big data distributed files [3].

• Random read/write IOPS is close to 95% of its physical disk performance, providing stable and efficient small file read/write performance.

• The instance can provide up to 30Gb/s of intranet bandwidth to meet the massive data interaction requirements between big data services.
  
  

03 Implementation mode

We combine the mount driver (virtio blk) with the performance enhancement method (iothread binding) to achieve Huawei ECS supports local data disks. The work includes but is not limited to:

1) The life cycle management of local data disks is designed and developed.

2) A dual factor hybrid scheduling algorithm based on the number of drawers and disk size is implemented.

3) It realizes the rapid cleaning method of large capacity data disks in the cloud environment.

4) Support the mixed mounting of data disks and cloud disks, and enhance the read/write processing performance of multiple disks.

The specific process logic is shown in the figure below. We have enhanced the management of local disks at the bottom of the cloud platform from multiple perspectives, further encapsulating and implementing the local storage model on Huawei's cloud Huawei ECS type.

04 Features

1.  Mixed mounting of local data disks and cloud disks

Local storage type Huawei The ECS supports the system disk to use the cloud disk, and relies on the underlying image cache to achieve rapid distribution. The creation time is no different from that of ordinary ECS. In addition, cloud disk can provide system disk snapshot/backup function, with certain high availability.

Local storage type Huawei ECS also supports attaching data disks as cloud disks, providing a variety of storage types to meet the needs of different scenarios.

2.  Enhance the concurrent performance of multiple disks

When Huawei When the number of local data disks attached to ECS reaches a certain level, the disk read/write performance will degrade to a certain extent. For example, when 10 data disks are attached in the test, the sequential write performance of a single disk 512K is 245MB/s, but when ten disks are written simultaneously, the average write performance of each disk is 200MB/s, which will decline to varying degrees.

Further research found that because event triggering of virtual machines can only run in a single thread, multi-core advantages cannot be played. Therefore, the multi iothread mechanism [4] is introduced to reduce the interference of lock contention and other simulation functions, and handle and respond to the virtual machine IO events separately. After optimization is introduced, the concurrent sequential write performance of multiple disks is consistent with that of a single disk.

05 limitations

Due to the local storage type Huawei ECS uses the local data disk on the server as the main storage, and its flexibility will be limited to a certain extent:

① Migration/configuration change is not supported: because the data exists on the local data disk, the local storage type Huawei ECS does not support configuration change, hot migration, and downtime migration.

② High availability limitation: At present, only the snapshot/backup function of the system disk can be realized, The local data disk does not support high availability, and the upper application needs to use a high availability architecture to ensure data reliability.

06 follow-up

In the future, it is planned to enrich the specifications and categories to cover more big data application scenarios, not limited to: supporting more and larger SATA disks to increase the upper limit of storage capacity; Launch high-performance local SAS disks to meet MPP database, HBase, ES and other scenarios; Provide local SSD disks for services with high IOPS requirements. The local storage type will also be followed up Huawei Optimization of ECS related functions and performance, introduction of bad disk repair mechanism, etc., to provide customers with a better cloud experience.

The above is the analysis of the neutrino network What are the advantages and disadvantages of the new Huawei cloud storage ECS? I hope it can help you to consider your business comprehensively. Huawei Cloud, a micro network agent, is an excellent cloud computing service provider. It can provide the best service for friends who want to rent Huawei ECS to open their own online business. It has perfect industry solutions and excellent cloud computing technology. Self developed pure The SSD architecture ECS, with 50000 IOPS random read/write speed and 800 Mb/s throughput as performance indicators, broke the industry record. ECS elastic ECS adopts pure SSD architecture hardware devices can easily obtain and enable the cloud in just a few minutes to meet your computing needs. Expand the flexibility of resource use, save computing costs and simplify IT operation work enables you to pay more attention to the innovation of core business.