The title of the invention: asynchronous remote copy system, and, storage control method
Technical field
[0001]
　The present invention relates to storage control of the storage system that performs remote copy is a copy of data between storage devices.
BACKGROUND
[0002]
　Remote copy is known which is a copy of data between the first and second storage devices. The remote copy, for example, synchronization is remote copy and asynchronous remote copy. If the first storage apparatus receives a write request from the host, if the synchronous remote copy is employed, after the data according to the write request is transferred from the first storage system to the second storage apparatus, the light the response of the completion is transmitted to the host device, if asynchronous remote copy is adopted, even if it is not transferred to the second storage device from the data first storage device according to the write request, the completion of the write response is sent to the host device.
[0003]
　Asynchronous remote copy system in which a remote copy is employed (hereinafter, asynchronous remote copy system) in, for example, the first storage device, in addition to the first DVOL (logical volume in which data is stored), the first JVOL I have (Journal (hereinafter, JNL) is a logical volume to be stored). Similarly, the second storage device, in addition to the second DVOL, having a second JVOL. The first storage device, when receiving the write request and stores the data according to the write request to the first DVOL, and stores the JNL of the data to the first JVOL. The first storage device, the JNL in the first JVOL, is transferred to the second storage apparatus, the second storage apparatus, the JNL from the first storage device, it writes the second JVOL. Then, the second storage apparatus, the data JNL in the second JVOL has, writes the second DVOL. Thus, data written in the first DVOL, it will be copied to the second DVOL.
[0004]
　Patent Document 1, an asynchronous remote copy system in a multi-target system is disclosed. In a multi-target system, more than one destination exists for one copy. Specifically, Patent Document 1 discloses for example the following.
(*) A first storage device, the second storage apparatus, and a third storage apparatus is.
(*) A first storage device is a copy source, the second and third storage apparatus, the copy destination from the first storage apparatus.
(*) And the first host device is connected to the first storage apparatus, the second storage apparatus and the second host device is connected to the third storage apparatus a third host device is connected that has been.
(*) The first storage device has a first DVOL, the first die 1JVOL, and a second die 1JVOL.
(*) Second storage device has a second DVOL, and a second JVOL.
(*) The first and second DVOL are paired. First DVOL is, is the primary of DVOL, second DVOL is a secondary DVOL.
(*) The third storage apparatus includes a third DVOL, and a third JVOL.
(*) The first and third DVOL are paired. First DVOL is, is the primary of DVOL, third DVOL is a secondary DVOL.
(*) The first host device, writes data to the first DVOL.
(*) A first storage device, the JNL data to be written into the first DVOL, written to both the first die 1JVOL and second die 1JVOL. JNL, in addition to the input and output target of the data of the host device, including the update number.
(*) JNL in the first die 1JVOL is copied to the second JVOL.
(*) The second storage device, among the one or more unreflected in JNL in the second JVOL, the data included in the second reflecting the DVOL (JNL JNL having the oldest update number, the Write to 2 of DVOL).
(*) JNL in the second die 1JVOL is copied to the third JVOL.
(*) Third storage device, a third of the one or more unreflected in JNL in the JVOL of the data included in the reflected to the third DVOL (JNL JNL having the oldest update number, the Write to 3 of DVOL).
[0005]
　Furthermore, according to Patent Document 1, when a failure in the first storage unit occurs, the second storage device becomes the copy source, the third storage device to copy.
CITATION
Patent literature
[0006]
Patent Document 1: JP 2006-65629 JP
Summary of the invention
Problems that the Invention is to Solve
[0007]
　After the second storage device is a copy source, the second host device, writes data to the second DVOL, the second storage apparatus, updates the update number, and including the update number data JNL I write in the second JVOL. JNL is copied to the third JVOL from second JVOL, third storage device, writes the data JNL in the third JVOL has the third DVOL.
[0008]
　Here, the origin of the update number in the second storage apparatus, it is the latest update number of the reflection already JNL in the second storage apparatus can be considered. This is because the second DVOL is because with a data up to the order indicated by their most recent update number.
[0009]
　However, it is undesirable case. That case, for example, the latest update number of JNL reflected already in the second storage apparatus, an old case than the most recent update number of the reflection already JNL in the third storage apparatus. This is because, in that case, those of the state of the second DVOL is in old state than the state of the third DVOL, it is because so that the data is copied to the DVOL the new state from the DVOL the old state.
[0010]
　Thus, when a failure in the first storage unit occurs, there are cases where operations are not properly continuous.
[0011]
　These problems, if the first storage apparatus is stopped for reasons other than disorders (eg, if the first storage device is stopped by the so-called planned outages for maintenance) also Arieru for.
[0012]
　An object of the present invention, in the asynchronous remote copy system in multi-target method is that the first storage device to properly continue the job be stopped.
Means for Solving the Problems
[0013]
　First and storage device connected to the first host device comprises a first storage resource groups, a second storage device connected to a second host device having a second storage resource groups When I and a third storage apparatus connected to the third host system has a third storage resource groups.
[0014]
　The first storage resource groups comprises a first data volume is a logical volume in which data is to be written, and a first journal storage resource is a storage resource journal data is written.
[0015]
　Second storage resource groups comprises a second data volume is a logical volume in which data is to be written, and a second journal storage resource is a storage resource journal data is written.
[0016]
　Third storage resource groups comprises a third data volume as a logical volume in which data is to be written, and a third journal storage resource is a storage resource journal data is written.
[0017]
　The first storage device, when the data is written into the first data volume from the host device updates the update number is a number that is updated every time data is written to the data volume in the first storage resource groups creates a journal containing the update number and its data, and writes the journal in the first journal storage resource.
[0018]
　Asynchronous remote copy of the multi-target system is performed. In its asynchronous remote copy, the journal is, the data in the first transferred from the storage device to the second storage apparatus a first data volume that is reflected in the second data volume is written to the second data volume is, and its journal, the data in the first data volume is written to the third data volume can be reflected in the first storage third data volume are transmitted from the device to the third storage apparatus . Specifically, for example, in the asynchronous remote copying, the following is performed.
(A1) a journal is copied from the first journal storage resource in the second journal storage resource.
(B1) a second storage device, one or more journals of unreflected in the second journal storage resource, the order of the update numbers to reflect the second data volume.
(A2) the journal is copied from the first journal storage resource in the third journal storage resource.
(B2) a third storage device, one or more journals of unreflected in the third journal storage resource, the order of the update numbers, reflecting the third data volume.
[0019]
　If the first storage device is stopped, the following process is performed.
(X1) and the second update number included in the recently reflecting journal in the storage system, which is new or the recently reflected in the updated number of the journal has is determined in the third storage apparatus.
(X2) is determined in the new storage unit ((storage device having a new and update number, which is determined by x1)), whether there are one or more differences journal of the second and third storage devices . One or more differential journal, is one or more journals to journal having update number is new and is determined by (x1) from the journal having the next update number is new and not determined updated number (x1) .
(X3) If (x2) is determined in the affirmative, the old storage device from the new storage system (as in the new storage apparatus of the second and third storage apparatus without the storage device), one or more differential Journal There is copied.
(X4) old storage device, one or more of the difference journal, in the order of the update number, to reflect on the data volumes in the old storage device.
BRIEF DESCRIPTION OF THE DRAWINGS
[0020]
It shows the structure of an asynchronous remote copy system according to [1] Embodiment 1 of the present invention.
[Figure 2] it shows the path between the storage device.
[Figure 3] JNL group 112A in FIG. 6, of 112B and 112C status and mirror of I indicate the valid / invalid.
[Figure 4] status and mirror of JNL group 112B and 112C in FIG. 7 show a valid / invalid.
[Figure 5] status and mirror of JNL group 112B and 112C in FIG. 10 show a valid / invalid.
[6] it shows the outline of the asynchronous remote copy of the multi-target system.
[7] shows a portion of a process executed when a failure in the first storage 105A has occurred.
[Figure 8] shows an overview of the difference resync from the third storage 105C to the second storage 105B.
[9] shows an overview of the difference resync from the second storage 105B to the third storage 105C.
[Figure 10] it shows an overview of the continuation of the business after the difference resync complete.
[Figure 11] shows the configuration of a first storage 105A.
[12] shows the structure of the JNL.
[13] shows the structure of the JVOL115A.
[14] it shows the structure of the meta information.
[15] it shows the control information included in each storage.
[Figure 16] JVOL it shows the structure of a valid bit map 701A.
[Figure 17] DVOL it shows the structure of a valid bit map 702A.
[18] it shows the structure of the JVOL management table 703A.
[19] it shows the structure of a JNL group management table 704A.
[Figure 20] shows the structure of a pair management table 705A.
[21] This shows the flow of the difference management and formation copy.
[Figure 22] shows the flow of the write processing in the first storage 105A.
[23] I show the flow of read processing of JNL.
[Figure 24] it shows the flow of the reflection process of JNL.
[Figure 25] shows the flow of processing relating to the checking of the usage of the first JVOL115A.
[Figure 26] shows the flow of processing performed in the case where the first storage 105A fails.
[Figure 27] the difference resync shows an example of possible cases.
[28] the difference resync shows an example of impossible cases.
[Figure 29] shows the timing at which the writing is allowed of data from the second host 103B to the second DVOL113B.
In [30] Example 2 of the present invention, it shows a timing, write is permitted in the data from the second host 103B to the second DVOL113B.
In [31] Example 2 of the present invention, when the difference resync to the second storage 105B is made ​​from the third storage 105C, it shows the flow of the write processing in the second storage 105B.
In [32] Example 2 of the present invention, when the difference resync to the second storage 105B is made ​​from the third storage 105C, it shows the flow of a JNL reflection processing in the second storage 105B.
In [33] Example 3 of the present invention, when the difference resync to the second storage 105B is made ​​from the third storage 105C, shows the flow of the read process in the second storage 105B.
In [34] a fourth embodiment of the present invention, it shows the flow of processing relating to the determination of whether or not the transfer of the difference JNL.
[35] shows an example of a case in which the transfer exists unnecessary difference JNL.
EMBODIMENTS OF THE INVENTION
[0021]
　Referring to the drawings, a description will be given of several embodiments of the present invention.
[0022]
　In the following description, "xxx table", it is to explain the various types of information in the expression "xxx Bitmap", various information may be represented in a data structure other than the table or a bitmap . "Xxx table" to indicate that it is not dependent on the data structure, the "xxx Bitmap" may be referred to as "xxx information."
[0023]
　In the following description, as the identification information of the different subject, the numbers are mainly used, other types of identifying information in place of the number (for example, the name) may be employed.
Example 1
[0024]
　Figure 1 shows a configuration of an asynchronous remote copy system according to a first embodiment of the present invention. In the following description, the journal is described as "JNL", the data volume is a logical volume in which data is written is described as "DVOL" is described as "JVOL," a JNL volume is a logical volume JNL are written is, the host device is described as "host", the storage device is described as "Storage", it describes a controller as "DKC".
[0025]
　Three or more of the site, for example, the first site 101A, a second site 101B, and a third of the site 101C, is. Hereinafter, for clarity of explanation, reference numerals of the elements of the first site 101A has is the parent numbers and child combination of symbols "A", reference number of the elements that second site 101B has a parent number and a combination of the child symbol "B", the reference numerals of the elements of the third site 101B has is a combination of the parent number and a child symbol "C".
[0026]
　First site 101A includes a first storage 105A, and the first host 103A connected to the first storage 105A. First storage 105A has a first DKC111A, a first JNL group 112A. One of JNL group 112A includes a DVOL113A, and JVOL115A.
[0027]
　The second and third site 101B, and 101C have the same configuration as the first site 101A.
[0028]
　Storage 105A and 105B with each other via a dedicated line (or communication network), are physically connected. Storage 105B and 105C with each other also via a dedicated line (or communication network), are physically connected. Storage 105A and 105C each other also via a dedicated line (or communication network), are physically connected.
[0029]
　As the path for connecting the storage together, as shown in FIG. 2, a control path, there is a data transfer path. Control path is a path that requires the difference resync (see below). Specifically, for example, in the difference resync process is a path sequence number acquisition request (described later) flows. Data transfer path is a path that JNL flows. Control path and data transfer path, either, is the path that can be bi-directional communication.
[0030]
　Again refer to Figure 1.
[0031]
　There is a logical connection between the JNL group called "mirror". According to Figure 1, the connection between the JNL group 112A and 112B are mirror # 0 (a mirror which is assigned the number "0" (M0)), the connection between the JNL group 112A and 112C are mirror # 1 ( a mirror number "1" is assigned (M1)), the connection between the JNL group 112B and 112C are a mirror # is 2 (mirrors assigned the number "2" (M2)).
[0032]
　Hereinafter, an outline of the process according to the present embodiment. In the following description, JNL group 112A, 112B, and 112C are, unless otherwise described, I to be one, respectively.
[0033]
　In the remote copy system according to this embodiment, it has become asynchronous remote copy of the multi-target system is performed, the first site 101A is the active site. In this case, as shown in Figure 3, the status of the first JNL group 112A is "master", the status of the second and third JNL group 112B and 112C are "Restore". Status "master" means that it is a copy source. Status "Restore" is meant to be a copy destination. In FIG. 3 (and FIGS. 4 and 5 below), the dotted line of the mirror is invalid mirrors, the solid mirror is an effective mirror.
[0034]
　Therefore, as shown in FIG. 6, the first DVOL113A the primary DVOL (hereinafter, PVOL), and the second and third DVOL113B and 113C, the secondary of the DVOL (hereinafter, SVOL) is.
[0035]
　As shown in Figure 6, the first site 101A a production site, the first host 103A is, along with the execution of operations to write data to PVOL113A (S11). First storage 105A is a sequence number (hereinafter, SEQ #) is updated, and creates a JNL having a written in the SEQ # and PVOL113A the updated data, and writes the JNL created in the first JVOL115A ( S12). SEQ # is the first JNL group 112A data (first DVOL of JNL group 112A) is updated each time to be written (as for example incremented by one (or 1 decrement)) number.
[0036]
　Second storage 105B is, reads the JNL from the first JVOL115A, and writes the read JNL in the second JVOL115B (S21). Second storage 105B is a JNL of one or more unreflected in the second JVOL115B, in ascending order of SEQ #, it is reflected in SVOL113B (S22). Specifically, the second storage 105B writes data not reflected in the JNL in the second JVOL115B have the SVOL113B. As a result, the data written to PVOL113A is, it will be copied to the SVOL113B.
[0037]
　Third storage 105C is, reads the JNL from the first JVOL115A, and writes the read JNL in the third JVOL115C (S31). Third storage 105C is one or more of the unreflected in JNL in the third JVOL115C, in ascending order of SEQ #, is reflected in SVOL113C (S32). Specifically, the third storage 105C writes the data to a third unreflected in JNL in JVOL115C has the SVOL113C. As a result, the data written to PVOL113A is, it will be copied to the SVOL113C.
[0038]
　More than two destination JNL group may be present on one of the copy source JNL group.
[0039]
　According to this embodiment, the JVOL included in one copy source JNL group, which is common to a plurality of destination JNL group. In other words, the number of JVOL included in one of the copy source JNL group is independent of the number of destination JNL group. This is a copy of the JNL between the storage device (transfer) is, by the copy destination storage device reads the JNL from the JVOL in the copy source storage system is realized. According to this feature, it is possible to save the storage capacity consumed in the first storage 105A.
[0040]
　Now, as shown in Figure 7, the first site 101A a production site to a fault in the first storage 105A has occurred (S40). In this case, the production site is switched from the first site 101A to the second site 101B. Specifically, fail-over is performed from the first host 103A to the second host 103B is (S41). Second host 103B is to transmit a predetermined command to the second storage 105B (S42).
[0041]
　Second storage 105B receives a predetermined command. In this case, as shown in Figure 4, the mirror # 2 is enabled by the storage 105B and / or 105C, the status of the second and third JNL group 112B and 112C are temporarily, a "master / Restore" Become. For the judgment of S43, between the second storage 105B and the third storage 105C, and in order to exchange SEQ #. Status "master / Restore" means that it is also the destination are also copied. The status of the second and third JNL group 112B and 112C, until the difference resync (described later) is completed, a "master / Restore."
[0042]
　Second storage 105B, in response to a predetermined command, requests SEQ # recently reflected by the JNL in the third storage 105C has (hereinafter, SEQ # (3)) to the third storage 105C (Sequence It transmits the number acquisition request), in response to the request, and receives SEQ # (3) from the third storage 105C. Second storage 105B is, SEQ # and (3), SEQ # having recently reflected in the JNL in the second storage 105B (hereinafter, SEQ # (2)) I to determine which one is new with. Depending on the result of the judgment, the difference resync shown in FIGS. 8 and 9 are performed.
[0043]
　That is, if the direction of SEQ # (3) is SEQ # (2) than the new, as shown in Figure 8, the second storage 105B is, from the third JVOL115C, reads the one or more differential JNL, a second to write to the JVOL115B (S44-1). Wherein means "one or more differential JNL" is one or more of the JNL from the JNL having the following SEQ # of the SEQ # (2) to the JNL having the SEQ # (3). Second storage 105B is, one or more of the difference JNL in the second JVOL115B, previously from the journal having the following SEQ # of SEQ # (2), in ascending order of SEQ #, reflected in the second DVOL113B (S45-1). As a result, the data in the second DVOL113B is consistent with the third of the data in the DVOL113C at the time of the difference resync start.
[0044]
　On the other hand, if the direction of SEQ # (2) is newer than the SEQ # (3), as shown in Figure 9, the second storage 105B is, the one or more difference JNL in the second JVOL115B, third JVOL115C to write to (S44-2). Wherein means "one or more differential JNL" is one or more of the JNL from the JNL having the following SEQ # of the SEQ # (3) to the JNL having the SEQ # (2). Third storage 105C is one or more differential JNL in the third JVOL115C, first from the journal having the following SEQ # of the SEQ # (3), in the ascending order of SEQ #, is reflected on the third DVOL113C (S45-2). As a result, the data in the third DVOL113C is consistent with the data in the second DVOL113B at the time of the difference resync start.
[0045]
　Then, the second site 101B becomes operational site, business is I continue. Specifically, as shown in Figure 5, the status of the second JNL group 112B becomes the "master", the status of the third JNL group 112C is "restored". Therefore, as shown in Figure 10, the second DVOL113B becomes PVOL, the third DVOL113C is SVOL. Second host 103B is, to write data to PVOL113B (S51). Second storage 105B is, updates the SEQ #, creates a JNL having the data written in the SEQ # and PVOL113B, and writes the JNL created in the second JVOL115B (S52). Third storage 105C is, reads the JNL from the second JVOL115B, and writes the read JNL in the third JVOL115C (S53). Third storage 105C is, the JNL in the third JVOL115C, in ascending order of SEQ #, to reflect on SVOL113C (S54).
[0046]
　According to this embodiment, the second storage 105B is, obtains SEQ # (3) from the third storage 105C, both SEQ # and (3) SEQ # (2) is to determine whether new, the determination Results based on, and controls whether to transfer the second and third storage 105B and one or more differential JNL from either in either 105C. This makes it possible fault in the first storage 105A is to properly continue operations even occur. In addition, one or more differential JNL to be transferred, since only differences between the second and third storage 105B and 105C, it is possible to suppress decrease the number of JNL transferred.
[0047]
　The following, we will describe the present embodiment in detail. At that time, as will be explained taking as a representative example of elements in the first site 101A, unless otherwise described, the elements in the second and third sites 101B and 101C is in the first site 101A is the component substantially the same.
[0048]
　11 shows a configuration of a first storage 105A.
[0049]
　First storage 105A includes a first DKC111A, multiple RAID (Redundant Array of Independent (or Inexpensive) Disks) group (hereinafter, RG) and 900A. RG900A is composed of a plurality of HDD (Hard Disk Drive). Instead of the HDD, another physical storage device such as a flash memory may be employed. One or a plurality of logical volumes, is based on one of RG900A. Logical volume, for example, is a DVOL or JVOL mentioned above, either a, may be a real logical volume is a part of the storage space of RG900A, it is configured based on one or more RG900 real area from the pool (storage area composed of a plurality of real areas) may be dynamically allocated virtual logical volumes (virtual logical volume according Thin Provisioning technology).
[0050]
　DKC111A, a plurality of front-end interface device (hereinafter, FE-IF) and 610A, the back-end interface device (hereinafter, BE-IF) and 150A, cache memory (hereinafter, CM) and 620A, shared memory (hereafter, SM) and it includes a 640A, and one or more of the CPU that are connected to them (Central Processing Unit) 630A. Processing DKC111A is, CPU630A may be performed by executing one or more computer programs, but at least part of the processing may be performed by hardware circuitry.
[0051]
　A plurality of FE-IF610A, first host 103A, the second and third storage 105B and 105C are connected. DKC111A (CPU630A) via the FE-IF610A, the first host 103A, which communicates with the second and third storage 105B and 105C.
[0052]
　The BE-IF150A, multiple RG900A is connected. DKC111A (CPU630A), through the BE-IF150A, writes the write destination logical volume (for example, the first DVOL113A or first JVOL115A) data to have been based on the RG900A (or JNL).
[0053]
　CM620A the data written to RG900A (and JNL), and stores the data (and JNL) read from RG900A.
[0054]
　SM640A stores a variety of control information which is used to control the processing of DKC111A.
[0055]
　CPU630A controls the process DKC111A do.
[0056]
　The above is the structure of the description of the first storage 105A. Incidentally, DKC111A is not limited to the configuration shown in Figure 11 may have other configurations. The configuration of the second and third storage 105B and 105C has a configuration substantially the same as the first storage 105A. Configuration of DKC111B or 111C may be called something other than the configuration of the DKC111A.
[0057]
　Figure 12 shows the configuration of a JNL, FIG. 13 shows the configuration of a JVOL115A.
[0058]
　JNL is composed of meta information and data.
[0059]
　JVOL115A is composed of a meta area 1201A, a data area 1203A. Meta area 1201A stores the meta information, the data area 1203A stores data. In addition, the meta-region 1201A is, CM620A, etc., may also be present in another storage resource and RG900A.
[0060]
　Figure 14 shows a structure of a meta information.
[0061]
　Meta information is management information about the data contained in the JNL. Meta information includes, for example, the following
information, (*) SEQ
#, (*) write destination information (where the information indicating whether the data is to be written in the
DVOL) (*) PVOL # (copied DVOL
number), ( *) SVOL # (destination DVOL
number), (*) information (this information representing a position in the JVOL the data corresponding to the meta information is included in the JNL are written in the JVOL),
including.
[0062]
　Typically, the first DKC111A is, it manages the SEQ #. SEQ # is in each JNL group 112A. The first DKC111A, when writing the data into DVOL113A in certain JNL group 112A, and updates the SEQ # corresponding to the JNL group 112A. SEQ # may be, for example, are stored in the SM640A or other storage resource.
[0063]
　Failure occurred in the first storage 105A, if the second site 101B becomes operational site, the second storage 105B is, manages a SEQ # for each JNL group 112B.
[0064]
　As shown in Figure 15, SM640A is, as the control information, for example, JVOL effective bitmaps 701A, DVOL effective bitmaps 702A, JVOL management table 703A, JNL group management table 704A, the pair management table 705A, the differential bitmap 706A, and It stores the mirror bitmap 707A. Similarly, SM640B in the second storage 105B stores the control information 701B ~ 707B, SM640C in the third storage 105C stores the control information 701C ~ 707C. Hereinafter, it will be described as a representative control information 701A ~ 707A.
[0065]
　JNL group may, for example, can have up to 64 logical volumes. JVOL valid bitmap 701A, as shown in Figure 16, for each JNL group has 64 bits. If the n-th (n is 0 or an integer 63) logical volume JVOL115A of, n-th bit is on (for example, 1).
[0066]
　DVOL valid bitmap 702A, as shown in Figure 17, for each JNL group has 64 bits. If the n-th (n is 0 or an integer 63) logical volume JVOL115A of, n-th bit is on (for example, 1).
[0067]
　JVOL management table 703A is present in every JNL group 112A. Table 703A, as shown in Figure 18, for each JVOL115A, representing information representing the leading address of the meta area, and the information indicating the size of the meta area, and the information indicating the head address of the data area, the size of the data area I and an information. In other words, the table 703A, for each JVOL115A, from where to where is representing how far from where meta-region is a data area.
[0068]
　JNL group management table 704A includes information about the JNL group. Specifically, for example, the table 704A, as shown in Figure 19, for each JNL group, the following
information, (*) JNL JNL group representing the number of the group #
1001A, representing the status of the (*) JNL group Status
1002A, (*) This mirror represents the number of mirrors are present in the remote copy system according to Example #
1003A, (*) party represents the number of the party the JNL group JNL group #
1004A, (*) partner JNL group party storage # 1005A that represents the number of the storage device with a
( *) recently purged JNL purged been SEQ # 1006A that represents the SEQ # with the
( *) purgeable SEQ representing the SEQ # with the purge has been good JNL #
1007A, (*), the read pre SEQ # 1008A that represents the SEQ # of the read JNL having recently
to have.
[0069]
　Pair management table 705A has information concerning the pair between DVOL. Specifically, for example, the table 705A, as shown in Figure 20, for each DVOL113A, the following
information, (*) DVOL113A
DVOL represent the number # 1101A, number (*) JNL group 112A including the DVOL113A JNL group representing the
# 1102A, (*) DVOL113A of the destination of the copy destination
VOL represent the number of DVOL # 1103A, (*) DVOL113A and the pair of the copy destination of DVOL, status 1104A, representing the pair status of DVOL113A
the It has.
[0070]
　The difference bitmap 706A has every DVOL113A. DVOL113A is composed of a plurality of blocks. Differential bitmap 706A represents how DVOL113A throat block is updated. That is, the bit with the differential bitmap 706A corresponds to the block. Differential bitmap 706A, if the pair status of DVOL113A is given the status is updated. Specifically, for example, DKC111A is a pair status of a certain DVOL113A is a predetermined status, and, when writing the data to a certain block in a DVOL113A, the difference bitmap 706A corresponding to the DVOL113A, the bit corresponding to the block, it wants to change on (for example, 1).
[0071]
　Mirror bitmaps 707A represents what the mirror is disabled what mirror valid. Specifically, the mirror bitmap 707A has a plurality of bits corresponding to a plurality of mirrors. For example, typically, it mirrors # 0 and # 1 is disabled is valid mirror # 2 (see Fig. 6). In this case, bits corresponding to a mirror # 0 and # 1 is turned on (for example 1), the bit corresponding to the mirror # 2 is off (for example, 0). Also, if the first storage 105A failure occurs, a mirror # 0 and # 1 is invalid mirror # 2 is valid (see FIG. 10). In this case, bits corresponding to a mirror # 0 and # 1 are turned off, the bit corresponding to the mirror # 2 is on.
[0072]
　Hereinafter, the number of the processing performed in this embodiment.
[0073]
　.
[0074]
　Before the start of the asynchronous remote copying to the SVOL (remote copy using JNL) from the PVOL, it is necessary that the data included in the SVOL the data contained in the PVOL (contents of PVOL) (the contents of the SVOL) are consistent with each other. To that end, the formation copy is performed. The "initial copying" is the copy to match the contents of the PVOL and SVOL.
[0075]
　As shown in Figure 21, the status of PVOL113A and SVOL113B pairs, both the status PVOL113A and SVOL113C pairs, there may be a suspended (S2101-1, S2101-2). Specifically, the pair status of PVOL113A is "PSUS" (primary suspended), the pair status of SVOL113B and 113C becomes "SSUS". In this case, the first DKC111A is, resets the SEQ # corresponding to the JNL group 112A including the PVOL113A predetermined value (eg, zero).
[0076]
　In this case, the first DKC111A can also write data into PVOL113A, not create JNL having the data. First DKC111A, when data is written to a certain write-destination block of PVOL113A (S2102), if the pair status of the PVOL113A is "PSUS", in the difference bitmap 706A corresponding to the PVOL113A, write-destination block The bits corresponding to and turn on (for example, 1) (S2103). If PVOL113A pair status is "PSUS", each time data is written to the block of the non-updated in PVOL113A, S2103 is performed.
[0077]
　First DKC111A is, the first host 103A (or the first management terminal (not shown) is connected to the DKC111A), undergoes a formation copy instruction (S2111). Receipt of the formation copy instruction is the trigger of the start of the formation copy. Formation copy, for the first pair (PVOL113A and SVOL113B pairs) second pair (PVOL113A and SVOL113C pairs), may however be performed in parallel, in the present embodiment is not performed in parallel, It is sequentially performed. Therefore, it is possible to alleviate the concentration of access to PVOL113A.
[0078]
　For example, first, as shown in Figure 21, the initial copy is performed for the first pair. That is, the first of DKC111A is, I want to identify the block corresponding to the bits that are turned on in the difference bitmap 706A (block in PVOL113A corresponding to the difference bitmap 706A). The first DKC111A is to create a JNL having the data of the specified block, the JNL created and not stored in the first JVOL115A, it is written to the second JVOL115B (S2112). Second DKC111B is, to reflect the JNL in the second JVOL115B to SVOL113B (S2113). For all the updated blocks (blocks corresponding to the bits turned on) in PVOL113A, S2112 and S2113 are performed.
[0079]
　After the formation copy is complete for the first pair, formation copy is performed for the second pair. That is, the first of DKC111A is, create a JNL having the data of the updated block in PVOL113A, the JNL that you created, is not stored in the first JVOL115A, written in the third JVOL115C (S2115). Third DKC111C is, to reflect the JNL in the third JVOL115C to SVOL113C (S2116). For all of the updated block in PVOL113A, S2115 and S2116 is performed.
[0080]
　.
[0081]
　As shown in Figure 22, the first DKC111A receives a write request specifying a PVOL113A the first host 103A (S2201), to ensure a CM area (cache memory) in CM630A, the CM area secured , and writes the data (write target data) according to the write request (S2202). The first DKC111A is, at this point, may respond to the write completion to the first host 103A.
[0082]
　First DKC111A the data of the write target in the CM630A, to write to the PVOL113A (S2203).
[0083]
　First DKC111A updates the SEQ # corresponding to the JNL group 112A containing the PVOL113A (S2211). The first DKC111A creates a JNL (S2212), writes the JNL created in the first JVOL115A. JNL created in S2212 includes a meta information including the SEQ # (or pre-update SEQ #) updated in S2211, the data written in PVOL113A in S2203. Its data may be data read from PVOL113A, may be data remaining in the CM630A.
[0084]
　.
[0085]
　Second DKC111B is taken as an example that reading the JNL from the first JVOL115A, explaining the JNL lead processing.
[0086]
　As shown in Figure 23, the second DKC111B is, to calculate the read target SEQ # (S2301). "Read target SEQ #" is a SEQ # contained in the JNL to be read. Read target SEQ # is specifically, JNL group 112B of mirror # 0 to the corresponding leads already SEQ # 1008B (JNL group management table 704B in the information 1008B) value 1 is added to the value represented by (that is, is a next value) of lead already SEQ # 1008B value represented.
[0087]
　Second DKC111B is, it sends a read request to the first storage 105A (S2302). The read request and a read target SEQ # calculated in S2301, JNL lead source and become JVOL115A number (or LUN corresponding thereto (Logical Unit Number)) and. Number of JVOL115A, for example, is specified from the control information SM640B stores. Specifically, for example, the control information comprises a number of JVOL112A with the JNL group 112A corresponding to the JNL group 112B having SVOL113B.
[0088]
　First DKC111A receives a read request from the second DKC111B. The first DKC111A, based on the read request, from the read source JVOL115A, identifying the JNL having the read target SEQ # (S2303). The first DKC111A reads the specified JNL from the read original JVOL115A, the read JNL, via a data transfer path between the first and second storage 105A and 105B, and transmitted to the second DKC111B ( S2304).
[0089]
　The second DKC111B receives the JNL from the first DKC111A, the JNL received is written into the second JVOL115B (S2305). The second DKC111B is a value representing the read already SEQ # 1008B corresponding to the mirror # 0 of the JNL group 112B having SVOL113B, is changed to a value represented by the read target SEQ # (S2306). In other words, the second of DKC111B is, the value represented by the lead already SEQ # 1008B, adds 1.
[0090]
　.
[0091]
　Is taken to reflect the JNL in the second storage 105B in example describes the reflection process of the JNL.
[0092]
　As shown in Figure 24, the second DKC111B is a JNL having the oldest of the SEQ # of the one or more unreflected in JNL in the second JVOL115B, it reflects the SVOL113B (S2401). Specifically, the second of DKC111B is, the JNL containing 1 large SEQ # than the value JNL group 112B of mirror # 0 to purge possible SEQ # 1007B corresponding (information contained in the JNL group management table 704B 1007B) represents , is read from the second JVOL115B, the data included in the read JNL, to write to SVOL113B.
[0093]
　The second DKC111B changes the value represented by the purgeable SEQ # 1007B corresponding to the mirror # 0 of the JNL group 112B (S2402). Specifically, the second DKC111B adds 1 to the value represented by that purgeable SEQ # 1007B.
[0094]
　Second DKC111B notifies the value represented by the purge possible SEQ # 1007B after the update to the first DCK111A (S2403).
[0095]
　The first DKC111A the value represented by the purgeable SEQ # 1007A corresponding to the mirror # 0 of the JNL group 112A, is changed to a value that is notified from the second DKC111B (S2404).
[0096]
　.
[0097]
　Each DKC is, the JVOL in the storage having the DKC, (or irregularly) periodically, to check. Below, it, I will be described as the first storage 105A as an example.
[0098]
　First DKC111A is (or irregularly) periodically performs processing shown in FIG.
[0099]
　That is, the first DKC111A is (to volume of the first JVOL115A, the proportion of the total volume of one or more of the JNL in the first JVOL115A) first utilization JVOL115A whether exceeds a predetermined threshold it is determined (S2501). If the result of judgment is negative (S2501: NO), the process ends.
[0100]
　If the result of the determination is positive (S2501: YES), the first DKC111A, it is determined whether the purge possible JNL in the JNL in the first JVOL115A exists (S2502). Purge possible JNL is up to JNL from JNL of the following (A) (B).
(A) JNL group 112A to purge already SEQ is compatible # 1006A JNL including a value representing the (see FIG. 19) as the SEQ #.
A value representing the purge allows SEQ # 1007A corresponding to the mirror # 0 of the (B) JNL group 112A, one of the purge possible value represented by SEQ # 1007A corresponding to the mirror # 1 of JNL group 112A, the smaller value JNL containing as SEQ #.
[0101]
　If S2502 is determined in the affirmative (S2502: YES), the first DKC111A is to purge the purge possible JNL from the first JVOL115A (S2503). Thereafter, the first DKC111A performs S2501.
[0102]
　If the result of the determination in S2502 is negative (S2502: NO), the first of DKC111A will be suspend the neck of the pair (S2504). The "neck pair", a pair corresponding to one of the mirrors # 0 and the mirror # 1, it is better pair of values ​​representing the purge allows SEQ # 10007A is small. The reason the pair has a neck, in order considerably smaller than the smaller purgeable SEQ # 10007A towards purging possible large SEQ # 10007A for, because the number of purge can JNL is has become less .
[0103]
　If you suspend the neck of the pair, the number of purge possible JNL increases. Specifically, the purge can JNL is up to JNL including a value representing the purge allows SEQ # 10007A larger as SEQ #. For this reason, first of DKC111A, after S2504, perform the S2502 and S2503.
[0104]
　The first DKC111A the case of suspending the neck of the pair for the pair will make a difference management and resync (S2510). Specifically, so that the following is performed (in the description herein, the neck of the pair is to be a pair that corresponds to the mirror # 1).
(*) The first DKC111A, when writing data into PVOL113A, if the write destination block bit corresponding to the data (bit in the differential bitmap 706A corresponding to PVOL113A) is if turned off, the bit It turns on the.
(*) The first DKC111A at a predetermined timing (for example, when receiving a resync command from the first host 103A (or the management terminal)), perform a resync. Specifically, the first DKC111A is a JNL including the data in the block corresponding to the bit on, written in the third JVOL115C.
[0105]
　Note that to suspend the neck of the pair of the pair corresponding to a mirror # 0 and the mirror # 1, even after the S2502 and S2503, when the result of the determination in S2501 is affirmative, the S2502 the result of the determination of and it will be negative. In this case, the first DCK111A also suspends the remaining pairs. As a result, I will be both S2101-1 and S2101-2 is performed in Figure 21. In this case, since all the JNL in the JNL group 112A is purged possible JNL, if it is purged their JNL in S2503, the result of the determination in S2501 is always negative, the process ends.
[0106]
　.
[0107]
　As shown in Figure 26, if a failure in the first storage 105A is generated (S2601), the failover from the first host 103A to the second host 103B is performed (S2602).
[0108]
　The second host 103B is, the take-over command as a predetermined command is transmitted to the second storage 105B (S2603).
[0109]
　Next, the second DKC111B, in response to the takeover command, to update the various status (S2604). Specifically, for example, the second DKC111B performs updates below.
(*) DVOL113B and the pair status that corresponds to the mirror # 0, it will be updated to "SSWS". Second DKC111B can, if DVOL113B is SVOL, usually, but prohibits the writing of data from the second host 103B to DVOL113B, "SSWS" may be a DVOL113B is SVOL, writing of data There it means that it is permitted.
(*) DVOL113B and the pair status corresponding to the mirror # 1, and updates the "SSUS". "SSUS" means that the pair corresponding to the mirror # 1 is suspended. Thus, even if the pair status corresponding to the mirror # 0 has become "SSWS", prohibition of data writing to DVOL113B it is maintained from the second host 103B.
(*) I want to change the status of DVOL113B to "HOLD". "HOLD" means that it is waiting for the start of a difference resync.
(*) In the mirror bitmap 707B, and it turns off the bit corresponding to the mirror # 0, to turn on the bits corresponding to the mirror # 2.
[0110]
　Next, the second DKC111B from the third storage 105C, to obtain a purge possible SEQ # value represented (S2605). Specifically, for example, the second DKC111B is a value represented by (information 1007C in the JNL group management table 704C) JNL group 112C and the mirror # purgeable corresponding to 2 SEQ # 1007C, second and third via the control path between the storage 105B and 105C, and requests to the third storage 105C. Third DKC111C, in response to the request, a value representing the purge allows SEQ # 1007C corresponding to a JNL group 112C and the mirror # 2, via said control path, and notifies the second DKC111B.
[0111]
　Next, the second DKC111B compares the value represented by purgeable SEQ # 1007B corresponding to the JNL group 112B and the mirror # 2, the value represented by the acquired purgeable SEQ # 1007C (S2606). That is, the second DKC111B is, SEQ # (hereinafter, SEQ # (2)) which JNL that reflects recent second storage 105B has an, SEQ # with recently reflected by the JNL in the third storage 105C ( or less, compared with the SEQ # (3)).
[0112]
　Next, the second DKC111B, it is determined whether the difference resync is possible (S2607).
[0113]
　Here, the difference resync is possible cases and the difference resync to explain the impossible case. In this case, SEQ # (2) and SEQ # of the (3), a larger SEQ #, will be referred to as "SEQ # (large)", a smaller SEQ #, referred to as "SEQ # (small)". Further, among the second and third storage 105B and 105C, the storage having SEQ # (large), is called a storage (large) and the other storage means and the storage (small).
[0114]
　Difference resync is the possible cases, cases as shown in Figure 27, namely, the storage purged been SEQ # value represented corresponding to the JNL group and mirror # 2 (large) is the SEQ # (small) hereinafter Case It is. In other words, storage (large) is, SEQ from the following SEQ # of SEQ # (small) (SEQ # (small) SEQ # equal to +1) laden JNL (below, JNL (X)) # JNL including the (large) (hereinafter, JNL (Y)) is a case to have. Hereinafter, one or more of the JNL from JNL (X) to JNL (Y) (namely, SEQ # (SEQ # from the same SEQ # in the small) +1 (SEQ # are inclusive of one or more JNL up large)) and It says "one or more of the difference JNL".
[0115]
　On the other hand, the difference resync is impossible case, the case, such as shown in FIG. 28, ie, storage purge already SEQ # value represented by the corresponding to the JNL group and mirror # 2 in the (large) is, SEQ # (small) of the next SEQ # (SEQ # (small) equal to +1 SEQ #) is a case that is a value greater than. This is because the SEQ # are not contiguous.
[0116]
　Again reference to FIG. 26.
[0117]
　If S2607 the result of determination in the negative (S2607: NO), the second DKC111B performs a full copy (S2608). That is, the second DKC111B is, all the data stored in the SEQ # (2) and SEQ # in the storage with a larger SEQ # of one of (3) DVOL (eg DVOL113B), SEQ # ( 2) you want to copy to DVOL in the storage (for example DVOL113C) with the smaller SEQ # of the ones of the SEQ # (3). As a result, the contents of the content and DVOL113C of DVOL113B match.
[0118]
　If the result of the determination in S2607 is positive (S2607: YES), if the SEQ # (2) is greater than the SEQ # (3) (S2609: YES), the difference from the second storage 105B to the third storage 105C resync is performed (S2610). In other words, the difference resync shown in Figure 9 is performed. Specifically, the second DKC111B reads one or more differential JNL from the second JVOL115B, 1 more write requests differences JNL (third write request specifying JVOL115C) of the third storage 105C By sending the one or more differential JNL, writes in the third JVOL115C (Alternatively, the third DKC111C is, from the following SEQ # of the read request designating the second JVOL115B (SEQ # (3) sends a read request) that includes SEQ # (2), and in response, receives one or more differential JNL from the second storage 105B, or may be written to them one or more differential JNL the third JVOL115C ). Third DKC111C is, one or more of the difference JNL in the third JVOL115C, previously from JNL having the following SEQ # of SEQ # (3), in ascending order of SEQ #, reflected in DVOL113C. As a result, the contents of the DVOL113C matches the contents of the DVOL113B.
[0119]
　If the result of the determination in S2607 is positive (S2607: YES), if the SEQ # (2) is smaller than the SEQ # (3) (S2609: NO), the difference from the third storage 105B to the second storage 105C resync is performed (S2611). In other words, the difference resync shown in FIG. 8 are performed. Specifically, the second DKC111B is, transmits a third read request specifying the JVOL115C (SEQ # (the next read request including the SEQ # (3) from the SEQ # 2)), and in response thereto Te receives one or more differential JNL from the third storage 105C, and writes them one or more differential JNL the second JVOL115B (Alternatively, the third DKC111B is, one or more of the difference from the third JVOL115C reads the JNL, 1 more write request differential JNL By sending (write request designating the second JVOL115B) to the second storage 105B, or may be written to one or more differential JNL in the second JVOL115B) . Second DKC111B is, one or more of the difference JNL in the second JVOL115B, previously from JNL having the following SEQ # of SEQ # (2), in ascending order of SEQ #, reflected in DVOL113B. As a result, the contents of the DVOL113B matches the contents of the DVOL113C.
[0120]
　As shown in FIG. 29, the second of DKC111B is, if the difference resync of the S2610 or S2611 has been completed (S2901: YES), from the second host 103B to permit the writing of data to the second DVOL113B (S2902) . Specifically, the second of DKC111B is, the pair status corresponding to DVOL113B and mirror # 2, is updated to "SSWS" (ie write permission). Thereby, not only the pair status corresponding to DVOL113B and mirror # 0, because also became pair status corresponding to the mirror # 2 to "SSWS", the second host 103B of data to the second DVOL113B writing It can become. In the case where the difference resync of the S2610 or S2611 has been completed, the second of DKC11B is, the second DVOL113B the PVOL (third DVOL113C is SVOL).
[0121]
　After this, in the second site 101B, business is continued. In the second site 101B, the initial value of the SEQ # corresponding to the JNL group 112B is, SEQ # (2) and SEQ # (3) larger SEQ # of the next SEQ # of one of the (ie, SEQ # ( 2) and it is a SEQ # (1 added value to the SEQ # of the larger of 3)).
[0122]
　Second host 103B is, in response to operations, and sends a write request specifying a PVOL113B the second storage 105B. Second DCK111B is, the data according to the write request, write to CM630B, then, it writes to PVOL113B. The second DKC111B updates the SEQ #, creates a JNL including a written data SEQ # of the updated (or SEQ # before update) to PVOL113B, writes the second JVOL115B. The third DKC111C may, by sending a read request specifying the second JVOL115B the second storage 105B, receives the JNL in the second JVOL115B from the second storage 105B. Third DKC111C writes the JNK received in the third JVOL115C. Third DKC111C is, a third of the non-reflection of the JNL in JVOL115C, in ascending order of SEQ #, reflected in SVOL113C. By this series of processing, the data written to the PVOL113B, is copied to SVOL113C.
[0123]
　As described above, according to the first embodiment, acquires the switching destination site 101B the production site, the second storage 105B is, SEQ # with recently reflected by the JNL in the third storage 105C for (SEQ # (3)) is, compares the SEQ # (3), SEQ # recently reflected by the JNL in the second storage 105B has a (SEQ # (2)). The second storage 105B is, SEQ # (2) and a purge already SEQ # in the larger storage having a SEQ # of one of the SEQ # (3), SEQ # (2) and the SEQ # (3) based on the relationship between the SEQ # of the smaller one, it will determine whether or not the difference resync is possible. As a result the SEQ # of the judgment (2) and the SEQ # difference resync based on the relationship (3) is performed. Thus, the first storage 105A is also stopped due to causes such as failure, business is appropriately continued.
[0124]
　Furthermore, according to the first embodiment, the JVOL included in one copy source JNL group is common to a plurality of destination JNL group. In other words, the number of JVOL included in one of the copy source JNL group is independent of the number of destination JNL group. This makes it possible to save the storage capacity consumed in the first storage 105A.
Example 2
[0125]
　Hereinafter, we will explain the second embodiment of the present invention. At that time, differences from the first embodiment will be mainly explained, the common point omitted or be simplified description of the first embodiment (for this, the same applies to Examples 3 and 4 below).
[0126]
　In the second embodiment, even in the difference resync, writing of data to the second DVOL113B is permitted. Specifically, as shown in Figure 30, the second DKC111B is, when it is found that is YES in S2607 in Fig. 26 (S3001: YES), and from the second host 103B to the second DVOL113B Allow the writing of data (S3002). Specifically, the second DKC111B includes a pair status corresponding to the second DVOL113B and mirror # 0, both the pair status corresponding to the second DVOL113B and mirror # 2, is changed to "SSWS." . Incidentally, because "time was found to be YES at S2607" is when determined that one of the difference resync the S2610 and S2611 are performed.
[0127]
　However, this case is not any device, the following problems may occur. That is, in the difference resync from the third storage 105C to the second storage 105B, the block having the data written from the second host 103B (latest data) (block in DVOL113B), and the data in the difference JNL (old data) there is a possibility that the written.
[0128]
　Example 2 thereof in order to avoid the problem, when the difference resync from the third storage 105C to the second storage 105B that is made final and binding, the process shown in Figure 31 is executed.
[0129]
　That is, the second DKC111B is, when receiving a write request specifying a DVOL113B the second host 103B (S3101), and writes the write target data in the write destination block in DVOL113B (S3102). At that time, the second of DKC111B is, the write destination block, to manage as a non-reflecting destination of JNL (S3103). Specifically, for example, SM640B may have stored a non-reflecting management information reflecting the JNL represents an unnecessary write destination, the second DKC111B are the SM640B, the number of DVOL113B, blocks of DVOL113B Address ( For example, the LBA (Logical Block Address)), it will be registered in a non-reflecting management information.
[0130]
　As shown in Figure 32, the second DKC111B, when reading the JNL from the second JVOL115B, if reflected destination blocks the JNL (block in DVOL113B) is not a non-reflecting target (S3202: NO), a While reflects JNL (S3203), if the reflecting target block is insensitive destination of the JNL (S3202: YES), it does not reflect the JNL (S3204).
[0131]
　In S3202, the "whether reflected destination blocks the JNL is insensitive destination", for example, the address of the reflecting target block is whether or not it is registered in the non-reflecting management information described above.
[0132]
　In S3204, and "does not reflect the JNL", for example, it is to ignore the JNL. JNL, which is ignored, the second may also be left in JVOL115B, may be immediately removed from a second JVOL115B.
[0133]
　According to the second embodiment can be started operations earlier than in Example 1, and, the resulting may issue (block latest data has been written by DVOL113B in difference resync is a write enabled state , the old data in the differential JNL is thus written) it can also be avoided.
Example 3
[0134]
　In Example 3, even in the difference resync, second host 103B is permitted to read data from the second DVOL113B (in Example 1, the read is prohibited). Specifically, for example, the second DKC111B when the it becomes YES in S2607 of Fig. 26 has been found, that a state of being allowed to read data from the second DVOL113B.
[0135]
　However, this case is not any device, the following problems may occur. In other words, in the midst of the difference resync from the third storage 105C to the second storage 105B, data second host 103B reads from DVOL113B is, the latest data (data contained in the SEQ # is the largest difference JNL) there is a possibility the old data without.
[0136]
　Example 3, that in order to avoid the problem, when the difference resync from the third storage 105C to the second storage 105B that is made final and binding, the process shown in Figure 33 is executed.
[0137]
　That is, the second DKC111B is, until the difference resync is completed, when receiving a read request specifying the DVOL113B (S3301), securing a CM area CM630B (S3302), the difference resync the third storage it is determined whether the difference resync to the second storage 105B from 105C (S3303).
[0138]
　If the result of this determination is negative (S3303: NO), the second DKC111B reads data from the specified read-source block read request (block in DVOL113B), writes the read data in the CM area, and it transmits the data of the CM in the region to the second host 103B (S3307).
[0139]
　If S3303 is determined in the affirmative (S3303: YES), the second DKC111B, it is determined whether the latest JNL lead source block becomes and target is in the second JVOL115B (S3304 ). Here referred to as "latest JNL", among the one or more difference JNL second and third JVOL115B and 115C have, be any one or more of the JNL lead source block becomes and target is a JNL having the greatest SEQ # of one or more of the SEQ # with their one or more of the JNL. Therefore, for the sake of this judgment, the second DKC111B is, to grasp the SEQ # the difference JNL in the third DVOL115B has.
[0140]
　If the result of this determination is affirmative (S3304: YES), the second of DKC111B is, the latest JNL from the second JVOL115B reflected in the read-source block in the DVOL113B (S3306), then, do the S3307. In addition, JNL second DKC111B is, in the difference resync, a difference JNL the lead original block is a reflection destination, having the SEQ # smaller SEQ # with the above latest old difference JNL (latest JNL than JNL ) be acquired, the difference JNL, does not reflect the read-source block (eg, purging the JNL from the second JVOL111B). Also, S3304: the S3307 in the case of YES, the data transmitted to the second host 103B may, instead be provided from a second DVOL113B, may be provided from JNL.
[0141]
　If the result of the determination in S3304 is negative (S3304: NO), the second DKC111B may, by sending a read request including the SEQ # of the newest JNL in the third storage 105C, the newest JNL No. and received from the third storage 105C, and writes the newest JNL in the second JVOL115B. That is, the second DKC111B obtains the latest JNL from the third storage 105C in preference over other JNL. Thereafter, the second of DKC111B performs S3306 and S3307 described above. Incidentally, S3304: Even S3307 if NO, the data to be transmitted to the second host 103B may, instead be provided from a second DVOL113B, may be provided from JNL.
[0142]
　According to the third embodiment can be started operations earlier than in Example 1, and the latest data to be problems (second host 103C which caused by DVOL113B is a read enable status while the difference resync In that no old data will be provided) it can also be avoided.
Example 4
[0143]
　In Example 4, even if the transfer of all the differences JNL conducted between storage 105B and 105C, it is possible to perform differential resync.
[0144]
　For example, in the following S2606 in Fig. 26, the second DKC111B is not reflected JNL corresponding to the difference JNL determines whether present in the old storage (S3401).
[0145]
　Here, the "old storage", a storage 105B, or 105C, and a storage apparatus having a SEQ # (2) and the SEQ # smaller SEQ # of the ones of (3) as a purge possible SEQ #. A person who is not a former storage of the storage 105B and 105C, or less, it says "new storage".
[0146]
　The "present in unreflected JNL the old storage corresponding to the difference JNL", as shown in Figure 35, the JVOL in old storage, not reflected with a purgeable SEQ # greater SEQ # in the old storage JNL it is that exists.
[0147]
　If the old storage of the second storage 105B, for example, if there is at least one non-reflection of the JNL in the second JVOL115B, the result of the determination in S3401 is made to be positive.
[0148]
　Also, if the old storage is a third storage 105C, for example, the second DKC111B is, the third DKC111C, by querying the value represented by the SEQ # with the latest JNL in one or more non-reflection of JNL, it is possible to perform the determination in S3401. Specifically, the second DKC111B the first 3 SEQ # which is responded DCK111C (hereinafter answer SEQ #) and, by comparing the SEQ # (3) +1, is making a judgment of S3401 it can. Answer SEQ # is, if the SEQ # (3) +1 above, the result of the determination in S3401 is made to be positive.
[0149]
　If the result of the determination in S3401 is negative (S3401: NO), the S2607 of FIG. 26 is performed.
[0150]
　If the result of the determination in S3401 is positive (S3401: NO), the second DKC111B, it is determined whether or not reflected JNL in the old storage is sufficient (S3402). SEQ # with the latest non-reflected JNL in the old storage, if the SEQ # or with the latest JNL of the one or more differential JNL, the result of this determination is affirmative.
[0151]
　If this determination result is positive (S3402: YES), the DKC in the old storage, among the JNL of one or more non-reflected in the old storage, the JNL corresponding to one or more differential JNL, all to reflect on DVOL in the old storage (S3403). If, if not reflected JNL is Amare, DKC in the old storage may purge any excess JNL.
[0152]
　If the S3402 as a result of the determination of is negative (S3402: NO), S3404 is performed. That, DKC in the old storage acquires shortfall differential JNL a (JNL corresponding to the difference between the unreflected the JNL and one or more differential JNL in the old storage) from the new storage and the difference JNL, Old Stored in JVOL in storage. And, DKC in the old storage, the not reflected JNL and the difference JNL in JVOL in the old storage, all of which reflected in DVOL in the old storage.
[0153]
　Having thus described the several embodiments of the present invention, the invention is not limited to these examples, it is needless to say various modifications are possible without departing from the gist thereof.
[0154]
　For example, two or more may be combined with one of Example 1-4.
[0155]
　Further, for example, in the above description, as the update number, a sequence number (SEQ #) there is employed, regularly larger or other types of numbers may be employed that becomes smaller.
[0156]
　Also, for example, in Examples 1-4, when the first storage 105A is stopped, the second storage 105B in switch destination site operational, various processes (eg, the determination of whether Derutarishinku can ) While performing at least one of Examples 1-4, at least one of a variety of processes may be performed by the third storage 105C.
[0157]
　Also, for example, all the JNL has the SEQ # of the updated Alternatively, it may have a SEQ # of the previous update.
[0158]
　Also, for example, at least one of the storage 105A ~ 105C, there may be multiple of JNL group.
[0159]
　Also, for example, at least one storage 105A ~ 105C, and may have a logical volume other than the logical volume included in the JNL group.
[0160]
　Also, for example, JNL transfer between the storage device, a transfer due to the copy destination storage device reads the JNL from the JVOL in the copy source storage devices, the copy source storage devices in the storage apparatus of the copy destination JVOL and it may be either transfer by writing the JNL in.
Description of the code
[0161]
　101A ... the first storage apparatus 101B ... second storage device 101C ... the third storage apparatus

Claims
[Claim 1]
　An asynchronous remote copy system is a storage system that performs Asynchronous Remote Copy,
　the first storage device connected to the first host device comprises a first storage resource groups,
　the second storage resource group comprising a second storage device connected to the second host device,
　and a third storage device having a third storage resource groups
having,
　the first storage resource groups, the data is written having a first data volume is a logical volume, a first journal storage resource is a storage resource journal data is written,
　the second storage resource group, it is a logical volume in which data is written in the and two data volumes, and a second journal storage resource is a storage resource journal data is written,
　said third storage resource groups, and the third data volume as a logical volume in which data is to be written , and a third journal storage resource is a storage resource journal data is written,
　when the first storage device, the data is written from the host device to the first data volume, said first The journal storage resource to update the update number is a number that is updated each time data is written to the data volume in the group to create a journal containing the update number and its data, the journal of the first is written in the storage resource,
　is done asynchronous remote copy of the multi-target method, reflected in its asynchronous remote copy, the journal, the first being transferred from the storage device to said second storage apparatus the second data volume data in the first data volume and is the difference is written to the second data volume, and the journal is transferred from said first storage device to the third storage apparatus and the third data in the first data volume that is reflected in the data volume is written to the third data volume,
　(X) when said first storage device is stopped,
(x1) and the second storage and update number included in the journal recently reflected in the apparatus, wherein either the new or the third recently reflected by the update number of the journal has in the storage device is determined,
(X2) the second and third storage devices Of the the journal having the next update number of the new storage apparatus is a storage device with a new and updated numbers that are determined by (x1), update number, which has not been determined to be new in the (x1) ( whether there are one or more differences journals is one or more journals to journals with a new and judgment has been updated number x1) is determined,
(X3) wherein the result is affirmative judgment (x2) If, from said new storage system, the old storage device in the storage system is not the thin storage device of said second and third storage devices, wherein the one or more differential journals are copied,
(X4) the old storage device, wherein the one or more differential journals, the order of the update number, and reflects the data volume in the old storage system,
the asynchronous remote copy system.
[Claim 2]
　The system of claim 1,
　wherein the second storage device, wherein an old storage system, the third storage apparatus, wherein a new storage device,
　the second data volume, said first is a write-prohibited state in which the writing is prohibited for the data from the second host device,
　wherein (x2) when the result of judgment is affirmative, the second storage apparatus, the second data volume, the and writing can write enabled state of the data from the second host device,
　the second storage device, after the write enable state and the second data volume, the write destination of the second data volume When receiving the specified write request from the second host device,
and writes the data in accordance with (w1) the write request to the write destination,
(W2) and the write destination, is managed as a non-reflecting target difference journal,
　the In (x4), the second storage
device, (x41) reflect the destination of the difference journal in the second journal storage resources, to determine whether or not the non-reflection
destination, (x42) said (x41 If the result of the determination in is negative), reflecting the difference journal,
if (x43) above (x41) The results of the determination is affirmative, it does not reflect the difference journal,
the asynchronous remote copy system.
[Claim 3]
　The system of claim 1,
　wherein the second storage device, wherein an old storage system, the third storage apparatus, wherein a new storage device,
　the second data volume, said first reading out the data from the second host device, a read inhibit state, which is prohibited,
　the (x2) when the result of judgment is affirmative, the second storage apparatus, the second data volume, the a read enabled state capable of reading data from the second host device,
　the second storage device, after the second data volume as a read enabled state, a read source of the second data volume When receiving the specified read request was from the second host device,
the (R) of the differential journals lead source is reflected destination data for the target journal is a difference journal having the latest update number a, the second It is transmitted to the second host device,
asynchronous remote copy system.
[Claim 4]
　The system of claim 3,
　in the (R), said second storage device,
(r1) in the second and third journal storage resource, of the difference journals lead source is reflected away , the subject journal is a difference journal having the latest update number, and determines whether the second journal storage resources,
if (r2) is determined in the (r1) is positive, the Reflecting the target journal to the second data volume, and transmits the data to the target journal has the second host device,
if (r3) above (r1) is determined in the negative, the target journal, in preference to the other journal is obtained from the third storage apparatus, and writes the target journal to the second journal storage resources, reflecting the target journal to the second data volume and transmits the data to the target journal has the second host device,
the asynchronous remote copy system.
[Claim 5]
　The system of claim 1,
　(P) and the second or the third storage device, said journal storage resource in the old storage unit, whether there is a journal of unreflected corresponding to the difference journal it is determined,
　if the affirmative result of the determination in (Q) above (P), the old storage device, a journal of the unreflected corresponding to the difference journal, is reflected in its own data volume,
　the ( The difference journal corresponding to the journal of the unreflected to be reflected in the Q), wherein in place of (x3) and (x4), wherein (Q) is performed,
the asynchronous remote copy system.
[6.]
　The system of claim 1,
　wherein the first storage device,
to purge the journal possible common purged from the first journal storage resource for (H) said second and third storage resource groups,
(I) the utilization of the first journal storage resource, it is determined whether it exceeds a predetermined threshold,
(J) if the result is affirmative determination of (I), wherein the first storage resource Of the first and the relationship is the relationship between the group and the second storage resource groups, and the second relationship is the relationship between the said first storage resource groups third storage resource groups, wherein the It is released towards the relationship between the number of possible purge journal little from one journal storage resource,
(K) and allows purging of the said second and third storage resource groups according to the relationship that is not released out of the storage resource groups It was purged with a journal from the first journal storage resource,
　and utilization of the first journal storage resource, for the capacity of the first journal storage resources, and one or more in the first journal storage resource The proportion of the total capacity of the JNL, is,
asynchronous remote copy system.
[7.]
　The system of claim 6 wherein,
　when (L) is the first storage apparatus, the suspended pair, the data is written to the write destination in the first data volume, JNL having the data was not created to manage that has been updated to the write destination,
　(M) is the first storage apparatus, in response to a predetermined command, including the data in a write destination is updated Journal Create and the journal, and sends to the storage device having a data volume relating to the neck of the pair,
　(N) the storage device, writing the journal in its journal storage resource, in the journal storage resource a journal to reflect on its own data volume,
asynchronous remote copy system.
[8.]
　The system of claim 1,
　the case of the (X), the first being the host device made ​​failover to the second host device, a second host device, the second storage device transmits a predetermined command,
　the second storage apparatus receives the predetermined command in response to said predetermined command, performing said (x1) and (x2),
the asynchronous remote copy system.
[9.]
　The system of claim 1,
　wherein the second data volume is a write prohibition state is prohibited to write data from the second host device,
　said second storage apparatus, wherein (x4 After completion), the second data volume, and the write-enabled state capable of writing data from the second host device,
the asynchronous remote copy system.
[10.]
　The system of claim 1,
　wherein the first journal storage resource is common to the second and third storage resource groups
　in the above (D1), said second storage apparatus, the second reads the journal having the next update number of the latest update number from said first storage apparatus in the second journal storage resource, writes the read journal said second journal storage resources,
　in the (D2), and the third storage device, reads the journal having the next update number of the latest update number from said first storage apparatus in said third journal storage resource, the read journal in the third journal storage resource writing,
asynchronous remote copy system.
[11.]
　A storage control method for an asynchronous remote copy system is a storage system that performs Asynchronous Remote Copy,
　the asynchronous remote copy system,
　the first connected to the first host device comprises a first storage resource groups and a storage device,
　a second storage device connected to a second host device having a second storage resource groups,
　and is connected to the third host system has a third storage resource groups a third storage apparatus
having a,
　a first storage resource groups, the first journal storage resource is a memory resource and the first data volume is a logical volume in which data is to be written, the journal data is written Preparative and a,
　the second storage resource groups comprises a second data volume as a logical volume in which data is to be written, and a second journal storage resource is a storage resource journal data is written,
　The third storage resource groups comprises a third data volume as a logical volume in which data is to be written, and a third journal storage resource is a storage resource journal data is written,
　the first storage device, if the data is written from the host device to the first data volume, updates the update number is a number that is updated every time data is written to the data volume in the first storage resource groups, Create a journal containing the update number and its data, writes the journal to the first journal storage resource,
　the asynchronous remote copying of the multi-target type is carried out, in that the asynchronous remote copy, the journal, the data in the first data volume is written to the second data volume by being reflected on the first are transferred from the storage device to said second storage apparatus the second data volume and the journal data in the first data volume that is transferred from said first storage device to the third storage system is reflected to the third data volume is written to the third data volume ,
　the storage control method,
　when the first storage device is stopped,
(x1) and the update number contained in the recently reflecting journal in the second storage apparatus, reflected recently in the third storage apparatus and determining which one is new to the update number to the journal has,
(X2) the second and of the third storage apparatus, the new storage device is a storage device with an updated number that is determined to be new in the (x1) the said one or more is a one or more journals from the journal having the next update number is new and not determined update number to the journal having update number that is determined to be new in the (x1) at (x1) it is determined whether there is a difference Journal,
(X3) when the result is affirmative judgment (x2), the journal storage resources in the new storage unit, of said second and third storage devices The said journal storage resource in the old storage device in the storage system is not a new storage device to copy the one or more difference journal,
said the old storage system, wherein one or more differential journal (x4), and update number in the order of, and it reflects the data volume in the old storage unit,
and a storage control method.