一般来说这个计划能够处理大部分数据存储问题,而且冷热库运用的是相同的库结构,数据从热库搬迁到冷库时不用进行数据转化,而且代码部分改动较小。和数据库分区相同,咱们在实施这个计划前,需求考虑这几个问题:
常见的判别办法是,依据主表中的一个或几个字段来判别。比方在工单体系中,能够运用工单状况、客服究竟操作时刻来作为冷热数据的判别条件,将现已封闭的而且超越一个月的工单视为冷数据,其他的工单视为热数据。
在每次批改了数据后,都会触发实施冷热别离的代码。这种办法比较简单,每次只需求判别以下是否变成了冷数据即可,虽然能确保数据实时性,可是无法依照日期时刻来差异冷热数据,而且全部与数据批改相关的代码都要加上冷热别离代码。因而这种办法运用的较少,一般用在小型体系上。
这种办法需求创立一个新服务来监听数据库改动日志,一旦发现相关的表发生了变动就触发冷热别离逻辑。这种办法又分为两种子办法,一个是直接触发冷热别离逻辑,另一个是将表更的数据发送到部队里(能够是自定义的公共 List,也能够是MQ),订阅放从部队中获取到数据后实施冷热别离逻辑。这种办法的利益是与事务代码完全解耦,低推延,可是缺点和办法一相同无法依照日期来差异冷热数据,而且会呈现事务代码和冷热别离逻辑代码一起操作同一条数据的问题,也就是并发问题。
In general, this plan can handle most of the data storage problems, and the hot and cold storage uses the same library structure, the data is moved from the hot storage to the cold storage without data conversion, and the code part changes less. As with database partitioning, we need to consider these issues before implementing this plan:
How to distinguish hot and cold data;
How to trigger cold and hot data separation;
How to end the separation of hot and cold data;
How to use hot and cold data.
The following is to clarify these four aspects
3.2.1.1 How to determine whether the data is Hot or cold
The common way to judge is to judge according to one or several fields in the main table. For example, in the work order system, the status of the work order and the actual operation time of the customer service can be used as the discrimination conditions of cold and hot data, and the work order that is closed and exceeds one month is regarded as cold data, and the other work order is regarded as hot data.
In the identification of hot and cold data, we should follow the following principles:
Once the data is moved to the cold storage, it means that the transaction code can only query it.
Hot and cold data cannot be read together.
3.2.1.2 How to trigger cold and hot data separation
There are three ways to trigger the separation of hot and cold data: add the code that triggers the separation after the code of the correction operation, listen to the database change log, and scan the database on time. These three methods are explained one by one.
Add the code that triggers the hot and cold separation after the code for the correction action
Each time the data is corrected, the code that implements the hot and cold separation is triggered. This method is relatively simple, each time only need to determine whether the following has become cold data, although it can ensure that the data is real-time, but it can not be different according to the date and time of the cold and hot data, and all the code related to data correction must be added to the cold and hot separation code. Therefore, this method is used less and is generally used on small systems.
Listen for database change logs
This approach requires the creation of a new service to listen to the database change log and trigger the hot/cold separation logic when changes to the associated tables are detected. This method is divided into two sub-methods, one is to directly trigger the hot and cold separation logic, the other is to send the data of the table to the army (can be a custom public List, can also be MQ), and the subscription is to implement the cold and hot separation logic after obtaining the data from the army. The advantage of this method is that it is completely decouple from the transaction code and has low latency, but the disadvantage is that the same method can not differentiate hot and cold data according to the date, and it will present the problem that the transaction code and the cold and hot separation logic code operate the same data together, that is, the concurrency problem.
其他型号推荐: F3YD32-1P横河晶体管输出模块 | Bently内华达两线速度地震探测传感器 |
F3YD64-1P横河晶体管输出模块 | Bently内华达州两线速度地震探测传感器 |
F3YP12-0V横河晶体管输出模块 | Bently内华达两线速度地震探测传感器 |
20P41AD330RA0NNN | VT2000-52-A力士乐电子放大器 |
1769-SDN Allen-Bradley DeviceNet通信模块 | VT-VSPA1-11-B力士乐电子放大器 |
1734-CTM艾伦-布拉德利点I/O终端模块 | 力士乐CKK & CKR紧凑型模块 |
1769-L18ER-BB1B艾伦布拉德利CompactLogix 5370 L1控制器 | 力士乐CKK & CKR紧凑型模块 |
Allen-Bradley十六(16)通道24VDC分立输入模块 | 330703- 050-10-02-00 Bently内华达州3300 XL 11毫米接近探头 |
HD22010-3 EMERSON系列充电模块 | 330103-00-10-02-00 Bently内华达州3300 XL 8毫米接近探头 |
1794-PS13 Allen-Bradley Flex I/O电源 | Bently Nevada 3300 XL 8毫米接近探头,公制 |
PPD539A102 ABB AC 800PEC控制装置 | 33086-10-01 00 Bently Nevada 3300 XL 50毫米近距离探头 |