The present invention is related to a multi-tenant system,
and especially, to a multi-tenant system in which an equipment
whose tenant identification data cannot be recognized is connected.
[Background Art]
[0002]
In the cloud computing, a multi-tenant system is known
which shares a server apparatus and hardware devices such as
switches and a storage unit among a plurality of users. FIG. 1 is
a diagram showing a resource sharing level in the multi-tenant
system. According to the resource sharing level, there are use
forms such as IaaS (Infrastructure as a Service), PaaS (Platform
as a Service) and SaaS (Software as a Service) in the multi-tenant
system. In the conventional system (a single tenant system), the
hardware, the OS/middleware and the application software are
managed by a user. In the IaaS, a system vendor manages the
hardware and the user manages the application software and the
OS/the middleware. In the PaaS, the system vendor manages the
hardware and the OS/middleware and the user manages the
application software. In the SaaS, the system vendor manages the
hardware, the OS/middleware and the application software.
[0003]
The multi-tenant'system is often realized by using the
tunneling protocol such as GRE (Generic Routing Encapsulation).
The reason is in that tenant identification 'data can be added when
a packet is encapsulated. Also, when using the tunneling protocol,
there is an advantage that the number of tenants allowed to
produce is not limited, compared with a case of realizing the
multi-tenant system by using VLAN. FIG. 2 is a diagram showing
the multi-tenant system which uses the tunneling protocol. In FIG.
2, a tenant A network and a tenant B network are built up in DC
(Data Center) 1. In the same way, a tenant A network and a tenant
B network are built up in DC 2. The tenant A network of DC 1 and
the tenant A network of DC 2 are connected by using the tunneling
protocol. The gateway of DC 1 and the gateway of DC 2 carry out
the production of an encapsulated packet through addition of
header data and decapsulation of the encapsulated packet by
removing the addition data. At this time, the tenant
identification data is contained in the addition data of the
encapsulated packet in addition to the L3 header.
[0004]
There are needs to use SAN (Storage Area Network) as a
sharing storage of the tenant A network and the tenant B network
in the multi-tenant system environment like DC 1 or DC 2 in FIG. 2.
However, any equipment whose tenant identification data cannot be
recognized (for example, SAN owned by a system vendor and a user
as existing property) is not designed to receive the encapsulated
packet by the tunneling protocol. For this reason, the equipment
whose existing tenant identification data cannot be recognized
cannot read the tenant identification data in the encapsulated
packet and cannot be used for the multi-tenant system.
[0005]
On the other hand, the open flow network system is known
in which the transfer processing of a packet by switches and the
control processing of the switches are separated. The open flow
network system includes one or more switches and a controller
which instructs the packet transfer control of the switches. The
controller sets a flow entry to a flow table of the switch. The
switch transfers the packet based on the flow entry. The switch
transfers the packet to the controller when the- flow entry for the
processing of the packet is not in the flow table upon receiving
the packet (hereinafter, to be referred to as "Packet-In"). The
controller produces the flow entry based on the packet subjected
to Packet-In and sets the flow entry to the switch which transfers
the packet. The controller replies the packet (hereinafter, to be
referred to as Packet-Out) to the switch which has carried out
Packet-In after setting the flow entry to the switch that it does
a packet.
[0006]
As Patent Literature 1 which relates to the field of the
present invention, JP 2009-152953A is known. The Patent
Literature 1 discloses a gateway apparatus and a packet
transferring method which can be used when the multi-tenant system
is realized, by rewriting an application header.
[Citation List]
[Patent literature]
[0007]
[Patent Literature 1] JP 2009-152953A
[Summary of the Invention]
LPM._nm^MA .M3,.z.X&jr_:»i
A.r- v ueuftX ©. 1. - 1 * - 2.Q I 3 2fc '- 2 %
identification data cannot be recognized.
[0013]
The packet transferring method of the present invention
is a packet transferring method which is executed by the multitenant
system. The packet transferring method of the present
invention includes' transmitting a packet from a virtual machine
(VM) to an equipment whose tenant identification data cannot be
recognized; carrying out packet-in of the packet into a controller
by a first switch; calculating an out-bound route of the packet by
the controller; and setting flow entries used to transfer the
packet to switches on the out-bound route by the controller. In
the setting, the flow entry which prescribes an operation of
encapsulating the packet by adding addition data which includes
the tenant identification data of a virtual machine (VM) to the
packet is set to the first switch. Also, the flow entry which .
describes an operation of removing the addition data from the
encapsulated packet is set to the second switch. The packet
transferring method of the present invention further includes
encapsulating, by the first switch, the packet based on the flow
entry by adding the addition data which contains the tenant
identification data of the virtual machine (VM); receiving the
encapsulated packet by the second switch; removing the addition
data from the encapsulated packet based on the flow entry by the
second switch; translating, by the second switch, an source IP
address of the packet from which the addition data has been
removed, into an IP address managed by the second switch based on
an address translation table; and transmitting the translated
packet in which the source IP the address has been translated, to
the equipment whose tenant identification data cannot be
recognized.
[Effect of the Invention]
[0014]
According to the present invention, the multi-tenant
system can be provided which can use the equipment whose tenant
identification data cannot be recognized.
[Brief Description of the Drawings]
[0015]
The Object, effects, features of the above invention
would become clearer from the description of the exemplary
embodiments in conjunction with .the attached drawings.
[FIG'. 1]
FIG. 1 is a diagram showing a sharing level of a resource
in a multi-tenant system.
i f « _ «ML©.l«_Fl.*..... %*.&.- A;fi_*/|SI.4..__I5_:15
[FIG. 2]
FIG. 2 is a diagram showing a multi-tenant system using a
tunneling protocol.
[FIG..3]
FIG. 3 is a diagram showing the whole configuration of a
multi-tenant system according to a first exemplary embodiment of
the present invention.
[FIG. 4]
FIG. 4 is a block diagram of an open flow controller
(OFC) 1 in the multi-tenant system according to the first
exemplary embodiment of the present invention.
[FIG. 5]
FIG. 5 is a block diagram of an open flow switch (OFS) 2
in the multi-tenant system according to the first exemplary
embodiment of the present invention.
[FIG. 6]
FIG. 6 is an example of a flow table 25 in the first
exemplary embodiment of the present invention.
[FIG. 7]
FIG. 7 is an example of a MAC address table 26 in the
first exemplary .embodiment of the present invention.
[FIG. 8]
FIG. 8 is an example of a tenant management table 27 in
the first exemplary embodiment of the present invention.
[FIG. 9]
FIG. 9 is an example of a gateway management table 28 in
the first exemplary embodiment of the present invention.
[FIG. 10]
FIG. 10 is an example of an address translation table 29
in the first exemplary embodiment of the present invention.
[FIG. 11]
FIG. 11 is a flow chart of a packet transferring method
(out-bound) in the multi-tenant system according to the first
exemplary embodiment of the present invention.
[FIG. 12]
FIG. 12 is a diagram schematically showing a packet on an
out-bound route (virtual machine (VM) 4-1 -> SAN 6) in the packet
transferring method (out-bound) of FIG. 11.
[FIG. 13]
FIG. 13 is a flowchart of the packet transf'e'rring method
(in-bound) in the multi-tenant system according to the first
exemplary embodiment of the present invention.
[FIG. 14]
FIG. 14 is a diagram schematically showing a packet on an
in-bound route (SAN 6 -» virtual machine (VM) 4-1) in the packet
transferring method (in-bound) of FIG. 13.
[FIG. 15]
FIG. 15 is a block diagram of the open flow controller
(OFC) 1 in the multi-tenant system according to a second exemplary
embodiment of the present invention.
[FIG. 16]
FIG. 16 is a block diagram of the open flow switch (OFS)
2 in the multi-tenant system according to the second exemplary
embodiment of the present invention.
[FIG. 17]
FIG. 17 is a flow chart of the packet transferring method
(out-bound) in the multi-tenant system according to the second
exemplary embodiment of the present invention.
[Description of the Exemplary Embodiments]
[0016]
A multi-tenant system according to exemplary embodiments
of the present invention will be described below with•reference to
the attached drawings.
[0017]
[First Exemplary Embodiment]
(Configuration)
First, the configuration of the multi-tenant system of
the exemplary embodiments will be described. FIG. 3 is a diagram
showing the whole configuration of a multi-tenant system according
to a first exemplary embodiment of the present invention. The
multi-tenant system of the present exemplary embodiment includes
an open flow controller (OFC) 1, a first open flow switch (OFS) 2-
1, a second open flow switch (OFS) 2-2, a server apparatus 3, a
bridge 5 and a SAN 6. The server apparatus 3 includes a virtual
machine (VM) 4-1 and a virtual machine (VM) 4-2. The
configuration example of FIG. 3 has a simple configuration for the
description but apparatuses of an optional number may be connected
with the multi-tenant system. Also, in the present exemplary
embodiment, the description is made as the resources allocated to
the user are the virtual machines (VM) which operate on the server
apparatus but may be physical terminals such as personal computers.
.[0018]
Hereinafter, the first open flow switch (OFS) .2-1 and the
second open flow switch (OFS) 2-2 are intensively called the open
flow switches (OFS) 2, in case of not distinguishing between them.
Also, the virtual machine (VM) 4-1 and the virtual machine (VM) 4-
2 are intensively called the virtual machines (VM) 4 in case of
JLF: 13 H&fc. HI Q JL. ~ 1. "£, - Z-O 2. 4 ii.5- *- 1. *S
not distinguishing between them.
[0019]
FIG. 4 is a block diagram showing the open flow
controller (OFC) 1 in the multi-tenant system according to the
first exemplary embodiment of the present invention. The open
flow controller (OFC) 1 of the present exemplary embodiment
includes a route calculating section 11 and a flow entry setting
section 12. The route calculating section 11 calculates a
transfer route of the open flow switches (OFS) 2 based on the
received packet subjected to packet-in. The flow entry setting
section 12 sets flow entries which are necessary for transfer
control of the packets, to the open flow switches (OFS) 2 on the
transfer route calculated by the route calculating section 11.
[0020]
FIG. 5 is a block diagram of the open flow switch (OFS) 2
in the multi-tenant system according to the first exemplary
embodiment of the present invention. The open flow switch (OFS) 2
of the present exemplary embodiment includes a transfer processing
section 21, a tunneling determining section 22, a processing
section 23, an address translating section 24 and a storage
section 30. A flow table 25, a MAC address table 26, a tenant
management table 27, a gateway management table 28 and an address
translation table 29 are stored in the storage section 30".
[0021]
The transfer processing section 21 transfers packets
based on the tables stored in the storage section 30. The
tunneling determining section 22 determines whether or not the
packet received by the open flow switch (OFS) 2 has been
encapsulated. Also, the tunneling determining section 22
determines whether or not the packet received by the open flow
switch (OFS) 2 has to be encapsulated. The processing section 23
carries out the encapsulation of the packet received by the open
flow switch (OFS) 2 and decapsulation of the encapsulated packet.
The address translating section 24 translates a port number and an
IP address which is contained in the packet, based on the address
translation table 29. Note that the translation may be carried
out to only either of the IP address or the port number.
[0022]
Next, the tables stored in the storage section 30 will be
described.
[0023]
FIG. 6 is an example of the flow table 25 according to
the first exemplary embodiment of the present invention. The flow
table 25 manages the flow entry which contains an operation rule
_Q_. ^_q. CUB *_ja. _•___ ~ -a. -o. _&. a—[T^IT ~ '-a *-.«''" *'-"W-n' "-« **. -3 "'*•?• -$ T&f
to a received packet which is defined based on a "rule", an
"action" and "statistics". Data used to identify the packet is
set to the "rule". For example, it is data as a combination of a
VLAN ID and a packet transmission source IP address and so on.
The "action" defines how the open flow switch (OFS) 2 should
process the packet conforming to the data contained in the "rule".
For example, a process of transmitting to a node having a
predetermined IP address from a predetermined port of the open
flow switch (OFS) 2 and a process of discardi'ng a packet and so on
are defined in the "action". For example, by combining the "rule"
and the "action", the operation of discarding the packet can be
set to the open flow switch (OFS) 2 when the protocol number is
"1" (ICMP). The "statistics" are statistic data every flow entry.
For example, it is the number of transfer packets and the number
of transfer octets.
[0024]
FIG. 7 is an example of the MAC address table 26
according to the first exemplary embodiment of the present
invention. The MAC address table 26 manages a MAC address of an
equipment which is connected with the open flow switch (OFS) 2,
and a switch port number of the open flow switch (OFS) 2 with
which the equipment having the MAC address is connected. Note
that the MAC address may be a virtual MAC address which is
allocated to the virtual machine (VM) and so on.
[0025]
FIG. 8 is an example of the tenant management table 27
according to the first exemplary embodiment of the present
invention. The tenant management table 27 manages a MAC address
of the equipment which is connected with the open flow switch
(OFS) 2 and tenant identification (ID) data of a tenant system to •
which the equipment having the MAC address belongs. Note that the
MAC address may be a virtual MAC address which is allocated to a
virtual machine (VM). Also, it is not necessary for the virtual
machine (VM) to have the tenant identification data and a tenant
to which the virtual machine (VM) belongs may be managed by the
open flow switch (OFS) giving the tenant identification data to
the virtual machine (VM). .
[0026]
FIG. 9 is an example of the gateway management table 28
according to the first exemplary"embodiment of the "present" --— .
invention. The gateway management table 28 manages an IP address
of a gateway for every gateway ID. The gateway ID is an
identifier which identifies the gateway uniquely, and is allocated
to be incremented from "1" and "0".
I r 5 O S i H l 3i " 1 £ ~ 'JLQ 1, fk 3. 5s "• 3, &
[0027]
FIG. 10 is an example of the address translation table 29
according to the first exemplary embodiment of the present
invention. The address translation table 29 manages a destination
IP address, a destination port number, a destination tenant, an
s
allocated IP address, an allocated port number, a source gateway
and a source virtual machine (VM). . The destination IP address and
the destination port number indicate an IP address and a port
number of an equipment which is connected with the multi-tenant
system and whose tenant identification data cannot be recognized.
The destination tenant is a tenant which is connected with the
multi-tenant system and which is allocated to an equipment whose
tenant identification data cannot be recognized. By allocating a
plurality of tenants to one equipment whose tenant identification
data cannot be recognized, a plurality of tenant systems may share
the equipment whose tenant identification data cannot be
recognized. The allocated IP address and the allocated port
number are a source IP address and a source port number set when a
packet is transmitted to the equipment whose tenant identification
data cannot be recognized from the open flow switch (OFS) 2. The
source gateway is data used to identify the source open flow
switch (OFS) 2 of the packet having header data to be rewritten
based on the address translation table 29. The source virtual
machine (VM) is data used to identify the source virtual machine
(VM) 4 of the packet having the header data to be rewritten, based
on the address translation table 29. Note that the address
translation table 29 may manage only the destination IP address
and the allocated IP address when the destination port number and
the allocated port number are not used.
[0028]
(Operation)
Next', a packet transferring method in the multi-tenant
system of the present exemplary embodiment will be described.
[0029]
First, the packet transferring method (out-bound) will be
described. FIG. 11 is a flow chart of the packet transferring
method (out-bound) iri the multi-tenant system according to the
first exemplary embodiment of the present invention. FIG. 12 is a
diagram schematically showing a packet on the out-bound route (the
virtual machine (VM) 4-1 -> SAN 6) in the packet transferrxng •
method (out-bound) of FIG. 11. The operation when the first open
flow switch (OFS) 2-1 receives a packet transmitted from the
virtual machine (VM) 4-1 to the SAN 6 for the first time will be
described below.
1H gSj-IHtj. If 3.. "* ! '£.• ~ '<£ <5 2 •$ jLife : 2.. &
[0030]
(Step SI)
The virtual machine (VM) 4-1 belonging to the tenant A
system transmits a packet for SAN 6. Note that in FIG. 12, the
packet (src: virtual machine (VM) 4-1, dst: SAN 6) which is
transmitted to the first open flow switch (OFS) 2-1 from the
server apparatus 3 corresponds to the above packet. Here, "src:
VM 4-1" indicates that the source IP address is an IP address
which is allocated to the virtual machine (VM) 4-1. In the same
way, "dst: SAN 6" shows that•the transmission destination IP
address is an IP address allocated to the SAN 6.
[0031]
(Step S2)
The first open flow switch (OFS) 2-1 receives the packet
transmitted at the step SI. The first open flow switch (OFS) 2-1
compares a MAC address corresponding to the destination IP address
of the packet and the MAC address table 26. The first open flow
switch (OFS) 2-1 determines that an equipment corresponding to the
MAC address is connected with another open flow switch (OFS) 2,
when the MAC address is not managed in the MAC address table 26.
In this case, the first open flow switch (OFS) 2-1 determines that
it is necessary to encapsulate the packet.
[0032]'
The first open flow switch (OFS) 2-1 refers to the packet
to determine that the SAN 6 is not connected with the first open
flow switch (OFS) 2-1.
[0033]
(Step S3)
The first open flow switch (OFS) 2-1 carries out packetin
of the packet into the open flow controller (OFC) 1, in order
to inquire the transfer route of the received packet and a
destination IP address in encapsulating the packet to the open
flow controller (OFC) 1. Note that the first open flow switch
(OFS) 2-1 determines whether or not an inquiry to the open flow
controller 1 is necessary, based on the MAC address table 26 (the
step S2). The first open flow switch (OFS) 2-1 may determine that
the inquiry to the open flow controller (OFC) 1 is necessary,
without referring to the MAC address table 26, when the flow table
25 does not have any flow entry used to transfer the received
packet. .. - . - ::\:"-m".:.™:;::'.j^^:
[0034]
(Step S4)
The open flow controller (OFC) 1 specifies that the open
flow switch (OFS) 2 connected with the SAN 6 as destination of the
If* tj...3.£JLM.JL.. .S_i. ~. 1..'£._r./£-M.X.^.... STP^TTJ'^
packet subjected to the packet-in is the second open flow switch
(OFS) 2-2. The open flow controller (OFC) 1 calculates the outbound
route used to transmit the packet-in packet to the second
open flow switch (OFS) 2-2.
[0035]
(Step S5)
The open flow -controller (OFC) 1 sets a flow entry used
to transfer the packet to each of the switches on the out-bound
route calculated at the step S4. Note that in FIG. 12, only the
first open flow switch (OFS) 2-1 and the second open flow switch
(OFS) 2-2 are shown to simplify the description. However, a
plurality of open flow switches (OFS) 2 may exist between the
first open flow switch (OFS) 2-1 and the second open flow switch
(OFS) 2-2. In such a case, the flow entry is set to each of the
open flow switches (OFS) 2 between the first open flow switch
(OFS) 2-1 and the second open flow switch (OFS) 2-2.
[0036]
Also, the open flow controller (OFC) 1 sets the flow
entry which prescribes an operation of encapsulating the packet by
adding addition data.
[0037]
Also, the open flow controller (OFC) 1 sets the flow
entry which prescribes an operation of removing the addition data
(decapsulation) to the second open flow switch (OFS) 2-2 connected
with the equipment whose tenant identification data cannot be
recognized.
[0038]
(Step S6)
The open flow controller (OFC) 1 carries out packet-out
of the packet subject to the packet-in at the step S3, to the
first open flow switch (OFS) 2-1.
[0039]
(Step S7)
The first open flow switch (OFS) 2-1 produces an
encapsulated packet by encapsulating the packet by using the
addition data. For example, the addition data contains an L3
header (source IP address: open flow switch (OFS) 2-1, destination
IP address: open flow switch (OFS) 2-2) and the tenant
identification data (tenant A). Here, the "source IP address:
open flow switch (OFS) 2-1" means that the source IP ~address"=or'—-
the encapsulated packet is the IP address of the first open flow
switch (OFS) 2-1. In the same way, the "destination IP address:
open flow switch (OFS) 2-2" means that the destination IP address
of the encapsulated packet is the IP address of the second open
flow switch (OFS) 2-2.
[0040]
(Step S8)
The second open flow switch (OFS) 2-2 receives the
encapsulated packet from the first open flow switch (OFS) 2-1.
The second open flow switch (OFS) 2-2 records the IP addresses of
the first open flow switch (OFS) 2-1, the IP address of the source
virtual machine (VM) 4-1, and the tenant identification .data, •
which are contained in the encapsulated packet, in the address
translation table 29. Also, the second open flow switch (OFS) 2-2
allocates DUM to the source IP address (allocated IP address of
the address translation table 29) used for the transfer of data
contained in the encapsulated packet. The record of the address
translation table 29 corresponding to the encapsulated packet is
set to be shown in FIG. 12.
[0041]
(Step S9)
The second open flow switch (OFS) 2-2 removes the
addition data given at the step S7 based on the flow entry set at
the step S5.
[0042]
(Step S10)
The second open flow switch (OFS) 2-2 translates the
transmission source IP address into DUM based on the address
translation table 29 of FIG. 12. Note that in the address
translation table 29 of the second open flow switch (OFS) 2-2
shown in FIG. 12, the destination port number and the allocated
port number are not used. In this way, the packet may be
rewritten by using only the destination IP address, the
destination tenant, the allocated IP address and the source
gateway. That is, the second open flow switch (OFS) 2-2 rewrites
the packet by using a NAT function or an IP masquerade function.
When the NAT function is used, the second open flow switch (OFS)
2-2 outputs a unique IP address for every source virtual machine
(VM) from the plurality of IP addresses managed by the second open
flow switch (OFS) 2-2. When the IP masquerade function is used,
the second open flow switch (OFS) 2-2 outputs a unique set of an
IP address and a port number for every source virtual machine (VM)
from the plurality of IP addresses and the plurality of port
numbers managed by the second open flow switch (OFST"2"-"27"""'~"
[0043]
(Step Sll)
The second open flow switch (OFS) 2-2 transmits for the
SAN 6, the packet that the source IP address has.been rewritten at
_4J?.B_jB^AllA__a-*--JL&_T_£J3i J*J_UL
the step S9.
[0044]
(Step S12)
The SAN 6 receives the rewritten packet through the
bridge 5. Note that the SAN 6 may be connected with the second
open'flow switch (OFS) 2-2 without going through the bridge 5.
[0045]
Thus, the packet transferring method (out-bound) of the
present exemplary embodiment has been described. Note that the
above-mentioned description is made for the operation when the
first open flow switch (OFS) 2-1 receives the packet transmitted
from the virtual machine (VM) 4-1 to the SAN 6 for the first time
(the first packet). The second and subsequent packets received by
the first open flow switch (OFS) are transferred by the open flow
switches (OFS) 2"on the transfer route based on the flow entries
set at the step S5.
[0046]
Next, a packet transferring method (in-bound) will be
described. FIG. 13 is a flow chart of the packet transferring
method (in-bound) in the multi-tenant system according to the
first exemplary embodiment of the present invention. FIG. 14 is a
diagram schematically showing a packet on the in-bound route (SAN
6 -» virtual machine (VM) 4-1) in the packet transferring method
(in-bound) of FIG. 13. The operation when the second open flow
switch (OFS) 2-2 receives the packet destined to the virtual
machine (VM) 4-1 from the SAN 6 to for the first time will be
described below.
[0047]
(Step S21)
The SAN 6 transmits a packet to the virtual machine (VM)
4-1. Note that the packet (src: SAN 6, dst: DUM) which is
transmitted from the SAN 6 toward the second open flow switch
(OFS) 2-2 through the bridge 5 corresponds to the packet in FIG.
14.
[0048]
. (Step S22)
The second open flow switch (OFS) 2-2 refers to the
address translation table 29 to translate the destination IP
address from DUM to the virtual machine (VM) 4-1.
[0049] - •—• "•—-' -
(Step S23)
The second open flow switch (OFS) 2-2 encapsulates the
packet transmitted at the step S21 by using the addition data to
produce the encapsulated packet. The second open flow switch
(OFS) 2-2 refers to the source gateway of the address translation
table 29 to specify that the source gateway is the first open flow
switch (OFS) 2-1. The second open flow switch (OFS) 2-2 refers to
the destination tenant of the address translation table 29 to
specify the tenant data to be included in the addition data. The
second open flow switch (OFS) 2-2 makes the addition data include
an L3 header (source IP address: open flow switch (OFS) 2-2,
destination IP address: open flow switch (OFS) 2-1) and the tenant
identification data (tenant A).
[0050]
(Step S24)
The first open flow switch (OFS) 2-1 carries out packetin
of the encapsulated packet into the open flow controller (OFC)
1.
[0051]
(Step S25)
The open flow controller (OFC) 1 calculates the in-bound
route based on the encapsulated packet subjected to the packet-in.
[0052]
(Step S26)
The open flow controller (OFC) 1 sets the flow entries
for transferring the encapsulated packet to the switches on the
in-bound route calculated at the step S25. Note that only the
first open flow switch (OFS) 2-1 and the second open flow switch
(OFS) 2-2 are shown in FIG. 14 to simplify the description, but a
plurality of open flow switches (OFS) 2 may exist between the
first open flow switch (OFS) 2-1 and the second open flow switch
(OFS) 2-2. In such a case, the flow entries are respectively set
to the open flow switches (OFS) 2 between the first open flow
switch (OFS) 2-1 and the second open flow switch (OFS) 2-2.
[0053]
Also, the open flow controller (OFC) 1 sets the flow
entry which prescribes the decapsulating operation of removing the
addition data to the last open flow switch (OFS) 2 on the in-bound
route (the first open flow switch (OFS) 2-1 connected with the
virtual machine (VM) 4-1).
[0054] '
(Step S27)
The open flow controller (OFC) 1 carries out packet-out
of the encapsulated packet subjected to the packet-in at-.-the step
S24, to the second open flow switch (OFS) 2-2.
[0055]
(Step S28)
The second open flow switch (OFS) 2-2 transmits the
.SF.BL ^_fyLSxzMXErzz2BX¥-Tjrrr
encapsulated packet to the first open flow switch (OFS) 2-1.
[0056]
(Step S29)
The first open flow switch (OFS) 2-1 removes the addition
data added at the step S23 from the encapsulated packet based on
the flow entry set at the step S26. Also, the first open flow
switch (OFS) 2-1 refers to the tenant data contained in the
addition data to check the transmission destination of the
decapsulated packet that the addition data has been removed.
[0057]
(Step S30)
The first open flow switch (OFS) 2-1 transmits the
decapsulated packet, in which the addition data has been removed,
to the virtual machine (VM) 4-1. The virtual machine (VM) 4-1
receives the decapsulated packet in.which the addition data has
been removed.
[0058]
According to the present exemplary embodiment, the multitenant
system is provided which can use the equipment whose tenant
identification data cannot be recognized.
[0059]
Next, as an example of the multi-tenant system in which a
part of the configuration of the multi-tenant system according to
the first exemplary embodiment of the present invention is changed,
a second exemplary embodiment will be described.
[0060]
[Second Exemplary Embodiment]
(Configuration)
First, the configuration of the multi-tenant system of a
second exemplary embodiment will be described. Because the whole
configuration of the multi-tenant system of the present exemplary
embodiment is same as that of the multi-tenant system of the first
exemplary embodiment shown in FIG. 3, the detailed description is
omitted.
[0061]
FIG. 15 is a block diagram of the open flow controller
(OFC) 1 in the multi-tenant' system of the second exemplary
embodiment of the present invention. The open flow controller
(OFC) 1 of the present exemplary embodiment includes the route
calculating section 11, the flow entry setting section 12, a
tunneling determining section 13 and a storage section 16. The
storage section 16 stores a tenant management table 27 and a
gateway management table 28. The route calculating section 11
calculates the transfer route based on the packet subjected to the
packet-in from the open flow switch (OFS) 2. The flow entry
setting section 12 sets the flow entries which are necessary for
the transfer control of the packet, to the open flow switches
(OFS) 2 on the transfer route calculated by the route calculating
section 11. The tunneling determining section 13 determines
whether or not the packet received from the open flow switch (OFS)
2 is necessary to be encapsulated. Also, the tunneling
determining section 22 determines whether or not the packet
received from the open flow switch (OFS) 2 is necessary to be
encapsulated.
[0062]
FIG. 16 is a block diagram of the open flow switch (OFS)
2 in the multi-tenant system according to the second exemplary
embodiment of the present invention. The open flow switch (OFS) 2
of the present exemplary embodiment includes the transfer
processing section 21, the processing section 23, the address
translating section 24 and a storage section 30. The storage
section 30 stores the flow table 25, the MAC address table 26 and
the address translation table 29.
[0063]
The transfer processing section 21 transfers the packet
based on the respective tables which are stored in the storage
section 30. The processing section 23 carries out the
encapsulation of the packet received from the open flow switch
(OFS) 2 and decapsulation of the encapsulated packet. The address
translating section 24 translates an IP address and a port number,
which are contained in the packet, based on the address
translation table 29. Note that the translation may be only
either of the IP address or the port number.
[0064]
The configuration of each of the tables which are stored
in the storage section 16 of the open flow controller (OFC) 1 and
the storage section 30 of the open flow switch (OFS) 2 is the same
as that of each table in the first exemplary embodiment, and
therefore, a detailed description is omitted.
[0065]
(Operation)
Next, a packet transferring method in the multi-tenant
system of the present exemplary embodiment will be described.
Because the packet transferring method (in-bound) - in the -present.-—
exemplary embodiment is same as that of the first exemplary
.embodiment, only a packet transferring method (out-bound) will be
described.
[0066]
* r « .B[.S=LU..«.*. *?i - i.fi: - 4*?-i.*i. x o.;- is
First, the packet transferring method (out-bound) will be
described. FIG. 17 is a flow chart of the packet transferring
method (out-bound) in the multi-tenant system according to the
second exemplary embodiment of the present invention. Fig. 17 is
same as FIG. 12 with respect to the diagram schematically showing
the packet on the out-bound route (virtual machine (VM) 4-1 -» SAN
6) .
[0067]
(Step S31)
The virtual machine (VM) 4-1 which belongs to the tenant
system transmits a packet to the SAN 6 as an equipment whose
tenant identification data cannot be recognized. Note that in FIG.
12, the packet (src: virtual machine (VM) 4-1, dst: SAN 6) which
is transmitted to the first open flow switch (OFS) 2-1 from the
server apparatus 3 corresponds to the packet. Here, "src: virtual
machine (VM) 4-1" means that the source IP address is an IP
address which is allocated to the virtual machine (VM) 4-1. In
the same way, "dst: SAN 6" means that the destination IP address
is an IP address which is allocated to the SAN 6.
[0068]
(Step S32)
The first open flow switch (OFS) 2-1 receives the packet
transmitted at the step S31.
[0069]
(Step S33)
The first open flow switch (OFS) 2-1 carries out the
packet-in of the packet into the open flow controller (OFC) 1.
[0070]
(Step S34)
The open flow controller (OFC) 1 determines whether or
not the packet has to be encapsulated based on a destination MAC
address of the packet subjected to the packet-in at the step S33.
When the destination MAC address is not one of MAC addresses
managed by the first open flow switch (OFS) 2-1, the open flow
controller (OFC) 1 determines to have to transmit the packet
through other open flow switches (OFS) 2. When the open flow
controller (OFC) 1 determines to have to transmit the packet
through other open flow switches (OFS) 2, the open flow controller
(OFC) 1'determines that the packet has to be encapsulated.
[0071]
(Step S35)
The open flow controller (OFC) 1 specifies that the open
flow switch (OFS) 2 connected with the SAN 6 as the destination of
the packet subjected to the. packet-in is the second open flow
.?«• e&LJHx . ©i - i ' F i i i r i ^ T f
switch (OFS) 2-2. The open flow controller (OFC) 1 calculates the
out-bound route used to transmit the packet, subjected to the
packet-in, to the second open flow switch (OFS) 2-2.
•[0072]
(Step S36)
The open flow controller (OFC) 1 sets the flow entries
used to transfer the packet to the switches on the out-bound route
calculated at the step S35. Note that only the first open flow
switch (OFS) 2-1 and the second open flow switch (OFS) 2-2 are
shown in FIG. 12 in order to simplify the description. However, a
plurality of open flow switches (OFS) 2 may exist between the
first open flow switch (OFS) 2-1 and the second open flow switch
(OFS) 2-2. In this case, flow entries-are set to the open flow
switches (OFS) 2 between the first open flow switch (OFS) 2-1 and
the second open flow switch (OFS) 2-2.
[0073]
Also, the open flow controller (OFC) 1 sets a flow entry
which prescribes an operation of encapsulating the packet by
adding the addition data, to the first open flow switch (OFS) 2-1.
[0074]
Also, the open flow controller (OFC) 1 sets a flow entry
which prescribes a decapsulating operation of removing the
addition data to the second open flow switch (OFS) 2-2 connected
with the equipment whose tenant identification data cannot be
recognized.
[0075]
(Step S37)
The open flow controller (OFC) 1 carries out the packetout
of the packet subjected to the packet-in at the step S32, to
the first open flow switch (OFS) 2-1. When the open flow
controller (OFC) 1 determines that the first open flow switch
(OFS) 2-1 has to encapsulate the packet subjected to the packet-in
at the. step S34 and has to transfer the encapsulated packet, the
open flow controller (OFC) 1 make a Packet-Out message include the
packet and the tenant identification data of the packet.
[0076]
(Step S38)
The first open flow switch (OFS) 2-1 encapsulates the
packet transmitted at the step SI by using the addition data to
produce an encapsulated packet. The addition data-, contains an L3
header (source IP address: open flow switch (OFS) 2-1, destination
IP address: open flow switch (OFS) 2-2) and the tenant
identification data (tenant A). Here, the "source IP address:
open flow switch (OFS) 2-1" means that the source IP address of'
the encapsulated packet is an IP address of the first open flow
switch (OFS) 2-1. In the same way, the "destination IP address:
open flow switch (OFS) 2-2" means that the destination IP address
of the encapsulated packet is an IP address of the second open
flow switch (OFS) 2-2.
[0077]
(Step S39)
The second open flow switch (OFS) 2-2 receives the
encapsulated packet from the first open flow switch (OFS) 2-1.
[0078]
(Step S40)
The second open flow switch (OFS) 2-2 removes the
addition data added at the step S37 based on the flow entry set to
at the step- S36.
[0079]
(Step S41)
The second open flow' switch (OFS) 2-2 translates the
source IP address into DUM based on the address translation table
29 of FIG. 12.
[0080]
(Step S42)
The second open flow switch (OFS) 2-2 transmits toward
the SAN 6, the packet in which the source IP address has been
rewritten at the step S40.
[0081]
(Step S43)
The SAN 6 receives the rewritten packet through the
bridge 5. Note that the SAN 6 may be connected with the second
open flow switch (OFS) 2-2 without going through the bridge 5.
[0082]
According to the present exemplary embodiment, the multitenant
system is provided which can use an equipment whose
existing tenant identification data cannot be recognized. Note
that the description of the exemplary embodiments of the present
invention have been made by using the SAN as an example of the
equipment whose tenant identification•data cannot be recognized.
However, the equipment whose tenant identification data cannot be
recognized is not limited to the SAN and the present invention may
be applied to other equipments.
[0083] . . •
In the above, the exemplary embodiments of the present
invention have been described with reference to the attached
drawings. However, the present invention is not limited to the
above-mentioned exemplary embodiments and can be appropriately
—*-• *~-a.- Bs-ni! •ftr—fi •%=: fl' *L_J> —*- *—-a- -a ' -a- - -o - -u n—u -«• >o -a &_V J- .-a. >viV" " '" ' '" '
modified by a skilled person in the art in a range which does not
deviate from the scope of the present invention.
[Explanation of Reference Numerals]
[0084]
1: OFC (Open Flow Controller)
2: OFS (Open Flow Switch)
2-1: First OFS
2-2: Second OFS
3: Server apparatus
4, 4-1, 4-2: VM (Virtual Machine)
5: Bridge
6: SAN (Storage Area Network)
11: Route Calculating Section
12: Flow Entry Setting Section
13: Tunneling Determining Section
16: Storage Section
21: Transfer Processing Section
22: Tunneling Determining Section
23: Processing Section
24: Address Translating Section
25: Flow Table
26: Mac Address Table
27: Tenant Management Table
28: Gateway Management Table
29: Address Translation Table
30: Storage Section
[Document Name] Scope of Patent to be Claimed
[Claim- 1]
A multi-tenant system comprising:
a server apparatus on which a virtual machine with tenant
identification data given operates;
a plurality of switches, each of which comprises a
processing section which processes a packet based on a flow entry
•set to said switch; and
a controller configured to set the flow entry to each of
said plurality of switches,
wherein said plurality of switches includes a first
switch connected with said server apparatus and a second switch
connected with an equipment whose tenant identification data
cannot be recognized,
wherein said controller comprises a flow entry setting
section configured to set to said first switch, the flow entry
which prescribes an operation of encapsulating a packet which is
f F 8 llj^k,8JL___E2L1L," 1,'A ~.."<£l.Q:.i_.#._... iS-'- 3,J5! ~~ ' ~~~~~
transmitted from said virtual machine on said server apparatus, by
adding addition data which includes the tenant identification data,
and set to said second switch, the flow entry which prescribes a
decapsulating operation of removing the addition data from the
encapsulated packet, and
wherein said second switch comprises:
an address translating section configured to translate a
source address of the packet from which the addition data has been
removed by said processing section, into an IP address managed by
said second switch; and •
a transfer processing section configured to transfer the
packet, whose source address has been translated, for said
equipment.
[Claim 2]
The multi-tenant system according to claim 1, further
comprising: a tunneling determining section configured to
determine whether or not the packet subjected to the packet-in
should be encapsulated,
wherein said flow entry setting section carries out
packet-out of'a packet-out message which includes the packet and
the tenant identification data of the packet, and
wherein said processing section of said first switch
encapsulates the packet based on the packet-out message.
[Claim 3]
The multi-tenant system according to claim 1 or 2,
wherein said second switch stores in said address translation
table, an IP address of said first switch, a IP address of said
virtual machine as a transmission source and the tenant
identification data which are contained in the encapsulated packet
when the encapsulated packet is received,
wherein said flow entry setting section of said
controller sets to said second switch, the flow entry which
prescribes the operation of encapsulating the packet transmitted
from said equipment by adding the addition data containing the
tenant identification data, and sets to said first switch, the
flow entry which prescribes the decapsulating operation of
•removing the addition data from the encapsulated packet,
wherein said transfer processing section , of._ said second__,
switch receives a reply packet transmitted to said virtual machine
from said equipment,
wherein said address translating section of said second
switch translates a destination IP address of the reply packet
into the IP address of said virtual machine based on said address
translation table,
wherein said the processing section of said second switch
encapsulates the reply packet subjected to the translation of the
source IP address, to the encapsulated reply packet in which the
addition data containing the tenant identification data of said
virtual machine has been added, wherein an IP address of said
first switch of said address translation table is set as a
destination IP address of an L3 header of the addition data,
wherein said processing section of said first switch
removes the addition data from the encapsulated reply packet, and
wherein said transfer processing section of said first
switch transmits the reply packet in which the addition data has
been removed, to said virtual machine.
[Claim 4]
The multi-tenant system according to any of claims 1 to 3,
wherein said address translating section of said second switch
translates a source port number of the packet into a port number
managed by said second switch and translates a destination port
' number of the reply packet into a source port number before the
translation.
[Claim 5]
The controller which is used in the multi-tenant system
according to any of claims 1 to 4.
[Claim 6]
The switch which is used in the multi-tenant system
according to any of claims 1 to 4.
[Claim 7],
A packet transferring method in a multi-tenant system,
comprising:
transmitting a packet from a virtual machine (VM) to an
equipment whose tenant identification data cannot be recognized;
carrying out packet-in of the packet into a controller by
a first switch;
calculating an out-bound route of the packet by said
controller; '-.'.:."r -----• -..,---,-
setting flow entries for packet transfer into switches on
the out-bound route by said controller;
wherein said setting comprises setting to the first
switch, a flow entry which prescribes an operation of
jL..PjuL_.G'&l,H.A. . 5 1 " 3.•'£ " £ 5 1 # iJsLJiJLlI
encapsulating the packet by adding addition data which contains.
tenant identification data of said virtual machine (VM) to the
packets, and setting in said second switch, a flow entry which
prescribes a decapsulating operation of removing the addition data
from- the encapsulated packet;
encapsulating, by said first switch, the packet based on
the set flow entry by adding the addition data which contains the
tenant identification data of said virtual machine (VM);
receiving the encapsulated packet by said second switch;
removing by said second switch, the addition data from
the encapsulated packet based on the flow entry set in said second
switch;
translating a transmission IP address of the decapsulated
packet from which the addition data has been removed into an IP
address managed by said second switch based on an address
translation table; and
transmitting the translated packet to said equipment by
said second switch.
[Claim 8]
The packet transferring method according to claim 7,
further comprising:
determining by said controller, whether or not the
packet-in packet should be encapsulated; and
carrying out packet-out of a packet-out message which
contains the packet and the tenant identification data of the said
packet by said controller.
[Claim 9]
The packet transferring method according to claim 7 or 8,
further comprising:
recording in the address translation table by said second
switch., in., said receiving, an IP address of said first switch.,, an
IP address of a source virtual machine (VM), and the tenant
identification data which are contained in the encapsulated
packet;
transmitting a reply packet, from. .said... equipment, to... said
virtual machine (VM);
, translating a destination IP address into the IP address
of said source virtual machine (VM) based on the address :'~':'-:
translation table by said second switch;
encapsulating the reply packet by adding the addition
data which contains the tenant identification data of said virtual
machine (VM) by said second switch to produce an encapsulated
$ 2. ~ 1. '£. ~. 'A 9 i-1 . i C T 8
reply packet;
setting the IP address of said first switch to the
address translation table as a destination IP address of an L3
header in the addition data;
carrying out packet-in of the encapsulated reply packet
into said controller by said second switch;
calculating an in-bound route of the encapsulated reply
packet by said controller;
setting to the switches on the in-bound route, the flow
entries for transferring the encapsulated reply packet by said
controller;
setting a flow entry which prescribes an operation of •
removing the addition data, to said first switch;
carrying out packet-out of the encapsulated reply packet
to said second switch by said controller;
transmitting the encapsulated reply packet to said first
switch from said second switch;
receiving the encapsulated reply packet by said first
switch;
removing the addition data based on the flow entry by
said first switch; and
transmitting the reply packet having the addition data
removed, to said virtual machine (VM) by said first switch.
[Claim 10]
The packet transferring method according to any of claims
7 to 9, wherein said encapsulating the packet comprises
encapsulating the packet in which a source port number of the
packet is translated into a port number managed by said second
switch; and
wherein said encapsulating the reply packet comprises
encapsulating the reply packet in which a destination port number
of .the reply, packet into a source port number before the
translation.
[Claim 11]
A program.. t.o... make. .a. mul.t.i-tenant..s.ys.t.em....exe.c.u.te a...packet
transferring method according to any of claims 7 to 10.
[Document Name] Abstract
[Abstract]
[Problem] A multi-tenant system is provided which can use an
equipment whose tenant identification data cannot be recognized.
[Solving Means] The multi-tenant system of the present invention
is realized by the Tunneling protocol. The multi-tenant system of
the present invention includes a server apparatus on which a
virtual machine (VM) with tenant identification data operates; an
equipment whose tenant identification data cannot be recognized; a
plurality of switches configured to transfer a packet based on
flow entries; and a controller configured to set the flow entries
to the switches. The plurality of switches includes a first
switch connected with the server apparatus and a second switch
connected with the equipment whose tenant identification data
cannot be recognized. The second switch rewrites the header of
the packet sent from and received by the equipment whose tenant
identification data cannot b recognized, based on an address
translation table.
:r 3L M £ t H3L . Q. 1. - .!_* ~.:£MXAk Z& • 0L5.
H
Hi
a
G
m
r
3
1
FIG. 1
fti
IS*
CI
-
<
5
<
L3 HEADER TENANT A
ORIGINAL
PACKET
L3 HEADER TENANT B
ORIGINAL
PACKET
DC2
HEADER ADDED THROUGH ENCAPSULATION
FIG. 3
OFC
3/15
SERVER APPARATUS
VM (TENANT A)
VM (TENANT B)
-4-1
-4-2
2-2
SECOND
OFS
^L
BRIDGE
SAN
-XPULJDEklTl &t~x;i
4/15
FIG. 4
ROUTE CALCULATING
SECTION
FLOW ENTRY SETTING
SECTION
—11
—12
^Q-BEJ^t^.gl--_l ••£•- CTTqrTgTIg:
FIG. 5
5/15
TRANSFER PROCESSING
SECTION
TUNNEUNG DETERMINING
SECTION
PROCESSING SECTION
ADDRESS TRANSLATING
SECTION
-21
-22
-23
-24
30
STORAGE SECTION
FLOW TABLE
MAC ADDRESS TABLE
TENANT MANAGEMENT
TABLE
GATEWAY MANAGEMENT
TABLE
ADDRESS TRANSLATION
TABLE
—25
—26
—27
—28
—29
J , V %f. U C t . f t X. Cf: l i L - i 8 1 ^ l j
FIG. 6
6/15
RULE ACTION STATISTIC -25
FIG. 7
MAC
ADDRESS
SWITCH PORT NO. -26
IP O Q£L. M x. v* I, " JL'£. •* "£ Q-1. 4 1"&'~:~1"~§"
L
7/15
FIG. 8
MAC
ADDRESS TENANT ID DATA
FIG. 9
GATEWAY ID IP ADDRESS
iF..eL._D£Li-ij-.x__@.i.-..i.:e -:£&.&.$.. _ i 6 . - - is
H
i
b
i FIG. 10
IS*
I
hi m
N
01
!"
|H
m
DESTINATION IP
ADDRESS
DESTINATION
PORT NO.
DESTINATION
TENANT
ASSIGNED IP
ADDRESS
ASSIGNED
PORT NO. SOURCE GA
9/15
FIG. 11. C START D
TRANSMIT PACKET FROM VM 4-1
RECEIVE PACKET BY FIRST OFS
PACKET-IN TO OFC 1
CALCULATE TRANSFER ROUTE
SET FLOW ENTRY TO OFS 2
ON THE TRANSFER ROUTE
PACKET-OUT TO FIRST OFS
ENCAPSULATE PACKET BY FIRST OFS
RECEIVE PACKET BY SECOND OFS
DECAPSULATE BY SECOND OFS
REWRITE PACKET HEADER BY SECOND OFS
TRANSMIT PACKET TO SAN
RECEIVE PACKET BY SAN
~S1
~S2
-S3
-S4
-S5
-S6
-S7
-S8
~S9
-S10
-S11
-S12
C END J
_j,ry u c u n x _£*_*_.
• -0- • -O ft—0. -* 9M.. -O B—• •» - 5 7 " t ^ - -
£.. ~ CXt. X. cfc- X. O, - X QFIG.
12
29
dst
IP ADDRESS
SAN6
dst
PORT No.
- .
dst
TENANT
A
ASSIGNED
IP ADDRESS
DUM
ASSIGNE
PORT N
-
FIRST OFS
src:VM4-1
dst:SAN6
SECOND OFS
X
BRIDGE
src:OFS2-1
dst:OFS2-2
TENANT A src:VM4-1
dst:SAN6
src:DUM
dst:SAN6
SERVER APPARATUS
VM (TENANT A)
VM (TENANT B)
-4-1
-4-2
z
SAN
11/15
FIG. 15 > C START j
\ '
TRANSMIT PACKET FROM SAN 6
> '
REWRITE PACKET HEADER BY SECOND ODS
> '
ENCAPSULATE PACKET BY SECOND OFS
> '
PACKET-IN TO OFC 1
V
CALCULATE TRANSFER ROUTE
"
SET FLOW ENTRY TO OFS 2
ON THE TRANSFER ROUTE
^ •
PACKET-OUT TO SECOND OFS
> t
TRANSMIT PACKET TO FIRST OFS
•
> r
DECAPSULATE BY FIRST OFS
> i
TRANSMIT PACKET TO VN4-1
i
f EN
' )
-S21
-S22
-S23
-S24
-S25
-S26
-S27
-S28
-S29
-S30
FIG. 14
OFC
dst
IP ADDRESS!
SAN 6
dst
PORT No.
FIRST OFS
src:SAN6
dst:VM4-1
29
dst
TENANT!
ASSIGNED
IP ADDRESS
DUM
ASSIGNED
PORT No. G
O
SECOND OFS
-C±.
BRIDGE
src:OFS2-2
dst:OFS2-1
TENANT A src:SAN6
dst:VM4-1
src: SAN6
dst:DUM
SERVER APPARATUS
VM (TENANT A)
VM (TENANT B)
-4-1
-4-2
SAN
13/15
FIG. 15
ROUTE CALCULATING
SECTION
FLOW ENTRY SETTING
SECTION
TUNNEUNG DETERMINING
SECTION
i?
— 11
—12
—13
16
STORAGE SECTION
GATEWAY MANAGEMENT
TABLE
ADDRESS TRANSLATION
TABLE
—27
—28
14/15
FIG. 16
TRANSFER PROCESSING
SECTION
PROCESSING SECTION
ADDRESS TRANSLATING
SECTION
-21
-23
-24
STORAGE SECTION
FLOW TABLE
MAC ADDRESS TABLE
ADDRESS TRANSLATION
TABLE
30
—25
—26
—29
__x.i?.3L oj&i-HX..-Si.-_i.^L-:iJ5ti,^-....i,&.- x&.
15/15
FIG. 17
( START )
V
TRANSMIT PACKET FROM VM 4-1
V
RECEIVE PACKET BY FIRST OFS
"
PACKET-IN TO OFC 1
''
OFC 1 DETERMINE WHETHER
TO BE ENCAPSULATED
"
CALCULATE TRANSFER ROUTE
"
SET FLOW ENTRY TO OFS 2
ON THE TRANSFER ROUTE
"
PACKET-OUT TO FIRST OFS
"
ENCAPSULATE PACKET BY FIRST OFS
"
RECEIVE PACKET BY SECOND OFS
>'
DECAPSULATE BY SECOND OFS
>'
REWRITE PACKET HEADER BY SECOND OFS
"
TRANSMIT PACKET TO SAN
>'
RECEIVE PACKET BY SAN
I
C END ")
—S31
~S32
—S33
~S34
—S35
~S36
—^S37
—S38
—S39
—^S40
—S41
—S42
—S43
VERIFICATION
I, Minoru KUDOH of c/o KUDOH PATENT OFFICE, 6F, KADOYA BLDG., 24-10,
Minamiooi 6-chome, Shinagawa-ku, Tokyo, 140-0013, Japan
solemnly and sincerely declare:
That I have a thorough knowledge of Japanese and English languages and that
the attached pages contain a correct translation into English of the following
Japanese patent application.
International Patent Application No. PCT/JP2013/063603
Date of Application: May 15, 2013
Signed this 11th day of November, 2014
^
Minoru KUDOH
iFenssoiS":wi_.-x:a.--^mxM-.,.xst^ us
MULTI-TENANT SYSTEM, SWITCH, CONTROLLER AND
PACKET TRANSFERRING METHOD
Technical Field
5 The present invention is related to a multitenant
system, and especially, to a multi-tenant
system in which an equipment whose tenant
identification data cannot be recognized is connected.
10 Background Art
In the cloud computing, a multi-tenant system
is known which shares a server apparatus and hardware
devices such as switches and a storage unit among a
plurality of users. FIG. 1 is a- diagram showing a
15 resource sharing level in the multi-tenant system.
According to the resource sharing level, there are use
forms such as IaaS (Infrastructure as a Service), PaaS
(Platform as a Service) and SaaS (Software as a
Service) in the multi-tenant system. In the
20 conventional system (a single tenant system), the
hardware, the OS/middleware and the application
software are managed by a user. In the IaaS, a system
vendor manages the hardware and the user manages the
application software and the OS/the middleware. In
25 the PaaS, the system vendor manages the hardware and
the OS/middleware and the user manages the application
software. In the SaaS, the system 'vendor manages the
hardware, the OS/middleware and the application
software.
30 The multi-tenant system is often realized by
using the tunneling protocol such as GRE (Generic
Routing Encapsulation). The reason is in that tenant
identification data can be 'a^dd'eol'"""Wh'eh"'''a""Jp^c'Ke"1:"-~i"3~
encapsulated. Also, when using the tunneling protocol,
35 there is an advantage that the number of tenants
allowed to produce is not limited, compared with a
I r o _!3Ltl»j^iL__3LS^_m^J^Jl^J3.i-^; x 5a - J. &
- 2 .-
case of realizing the multi-tenant system by using
VLAN. FIG. 2 is a diagram showing the multi-tenant
system which uses the tunneling protocol. In FIG. 2,
a tenant A network and a tenant B network are built up
5 in DC (Data Center) 1. In the same way, a tenant A
network and a tenant B network are built up in DC 2.
The tenant A network of DC 1 and the tenant A network
of DC 2 are connected by using the- tunneling protocol.
The gateway of DC 1 and the gateway of DC 2 carry out
10 the production of an encapsulated packet through
addition of header data (hereinafter, to be referred
to as "addition data") and decapsulation of the
encapsulated packet by removing the addi.tion data. At
this time, the tenant identification data is contained
15 in the addition data of the encapsulated packet in
addition to the L3 header.
There are needs to use SAN (Storage Area •
Network) as a sharing storage of the tenant A network
and the tenant B network in the multi-tenant system
20 environment like DC 1 or DC 2 in FIG. 2. However, any
equipment whose tenant identification data cannot be
recognized (for example, SAN owned by a system vendor
and a user as existing property) is not designed to
receive the encapsulated packet by the tunneling
25 protocol. For this reason, the equipment whose
existing tenant identification data cannot be
recognized cannot read the tenant identification data
in the encapsulated packet and cannot be used for the
multi-tenant system.
30 On the other hand, the open flow network
system is known in which the transfer processing of a
packet by switches and the control processing of the
switches are separated. The open flow network system
includes one or more switches and a controller which
35 instructs the packet transfer control of the switches.
The controller sets a flow entry to a flow table of
- 3 -
the switch. The switch transfers the packet based on
the flow entry. The switch transfers the packet to
the controller when the flow entry for the processing
of the packet is not in the flow table upon receiving
5 the packet (hereinafter, to be referred to as "Packet-
In") . The controller produces the flow entry based on
the packet subjected to Packet-In and sets the flow
entry to the switch which transfers the packet. The
controller replies the packet (hereinafter, to be
10 referred to as Packet-Out) to the switch which has
carried out Packet-In after setting the flow entry to
the switch that it does a packet.
As Patent Literature .1 which relates to the
field'of the present invention, JP 2009-152953A is
15 known. The Patent Literature 1 discloses a gateway
apparatus and a packet transferring method which can
be used when the multi-tenant system is realized, by
rewriting an application header.
2 0 Citation List
[Patent Literature 1] JP 2009-152953A
Summary of the Invention
An object of the present invention is to
25 provide a multi-tenant system which can use an
equipment whose tenant identification data cannot be
recognized.
The multi-tenant system of the present
invention includes a server apparatus on which a
30 virtual machine (VM) with tenant identification data
operates, an equipment whose tenant identification
data cannot be recognized, a plurality of switches
configured to transfer a pa'ck"e"t "b'a's'ecl" on-TTow ••e-nifries,--
and a controller configured to set the flow entries' to
35 the plurality- of switches. The plurality of switches
includes a first switch connected with the server
± i r S : ttjSLLJ&jL J3LJL~ 3^£/^J£J*Li_#._.._x_S_-_jtJ£. . __^..^_-^^_-
apparatus and a second switch connected with the
equipment whose tenant identification data cannot be
recogni zed.
The first switch includes a processing
5 section which encapsulates the packet transmitted from
the virtual machine (VM) to the equipment whose tenant
identification data cannot be recognized, by adding
addition data which contains the tenant identification
data of the virtual machine (VM), to produce an
10 encapsulated packet, and a transfer processing section
which carries out packet-in of the encapsulated packet
into the controller and which transfers the
encapsulated packet subjected to packet-out from the
controller, based on the flow entry.
15 The controller includes a route calculating
section which calculates the out-bound route of the
encapsulated packet subjected to the packet-in and a
flow entry setting section which sets a flow entry to
each of the switches on the out-bound route. The
20 controller sets to the second switch, the flow entry
which prescribes.an .operation of removing addition
data from the encapsulated packet.
The second switch includes a transfer
processing section which receives the encapsulated
25 packet; a processing section which removes the
addition data of the encapsulated packet based on the
flow entry; and an address translation section which
translates a source IP address of the packet into an
IP address managed by the second switch based on an
30 address translation table. The transfer processing
section transmits the packet in which the source IP
address has been translated, to the equipment whose
tenant identification data cannot be recognized.
The packet transferring method of the present
35 invention is a packet transferring method which is
executed by the multi-tenant system. The packet
_AJEL L e j t j g L j t . ESLJL_
- 5 -
transferring method of the present invention includes
transmitting a packet from a virtual machine (VM) to
an equipment whose tenant identification data cannot
be recognized; carrying out packet-in of the packet
• 5 into a controller by a first switch; calculating an
out-bound route of the packet by the controller; and
setting flow entries used to transfer the packet to
switches-on the out-bound route by the controller. In
the setting, the flow entry which prescribes an
10 operation of encapsulating the packet by adding
addition data which includes the tenant identification
data of a virtual machine (VM) to the packet is set to
the first switch. Also, the flow entry which
describes an operation of removing the addition data
15 from the encapsulated packet is set to the second
switch. The packet transferring method of the present
invention further includes encapsulating', by the first
switch, the packet based on the flow entry by adding
the addition data which contains the tenant
20 identification data of the virtual machine (VM);
receiving the encapsulated packet by the second
switch; removing the addition data from the
encapsulated packet based on the flow entry by the •
second switch; translating, by the second switch, an
25 source IP address of the packet from which the
addition data has been removed, into an IP address
managed by the second switch based on an address
translation table; and transmitting the translated
packet in which the source IP the address has been'
30 translated, to the equipment whose tenant
identification data cannot be recognized.
According to the present invention, the
multi-tenant system can be provided wFi^fPc^arTuiFTlTe"
equipment whose tenant identification data cannot be
35 recogni zed.
•i,r\! mJLMX„.Mi^iJ2_- cili^ l.fc VJLfc.
- 6 -
Brief Description of the Drawings
The Object, effects, features of the above
invention would become clearer from the description of
the exemplary embodiments in conjunction with the
5 attached drawings.
FIG. 1 is a diagram showing a sharing level
of a resource in a multi-tenant system.
FIG. 2 is a diagram showing a multi-tenant
system using a tunneling protocol.
10 FIG. 3 is a diagram showing the whole
configuration of a multi-tenant system according to a
first exemplary embodiment of the present invention.
FIG. 4 is a block diagram of an open flow
controller (OFC) 1 in the multi-tenant system
15 according to the first exemplary embodiment of the
present invention.
FIG. 5 is a block diagram of an open flow
switch (OFS) 2 in the multi-tenant system according to
the first exemplary embodiment of the present
2 0 invention.
FIG. 6 is an example of a flow table 25 in
the first exemplary embodiment of the present
invention.
• FIG. 7 is an example of a MAC address table
25 26 in the first exemplary embodiment of the present
invention.
FIG. 8 is an example of a tenant management
table 27 in the first exemplary embodiment of the
present invention.
30 FIG. 9 is an example of a gateway management
table 28 in the first exemplary embodiment of the
present invention.
FIG. 10 is an example of an address ~ ~~
translation table 29 in the first exemplary embodiment
35 of the present invention.
- 7 -
FIG. 11 is a flow chart of a packet
transferring method (out-bound) in the multi-tenant
system according to the first exemplary embodiment of
the present invention.
5 FIG. 12 is a diagram schematically showing a
packet on an out-bound route (virtual machine (VM) 4-1
-> SAN 6) in the packet transferring method (out-bound)
of FIG. 11..
FIG. 13 is a flow chart of the packet
10 transferring method (in-bound) in the multi-tenant
system according to the first exemplary embodiment of
the present invention.
FIG. 14 is a diagram schematically showing a
packet on an in-bound route (SAN 6 -» virtual machine
15 (VM) 4-1) in the packet transferring method (in-bound)
of FIG. 13.
FIG. 15 is a block diagram of the open flow
controller (OFC) 1 in the multi-tenant system
according to a second exemplary embodiment of the
20 present invention.
FIG. 16 is a block diagram of the open flow
switch (OFS) 2 in the multi-tenant system according to
the second exemplary embodiment of the present
invention.
25 FIG. 17 is a flowchart of the packet
transferring method (out-bound) in the multi-tenant
system according to the second exemplary embodiment of
the present invention.
30 Description of the Exemplary, Embodiments
A multi-tenant system according to exemplary
embodiments of the present invention will be described
below with reference to the att"a"cFTe d"~3rawlncfs . '""•"
35 [First Exemplary Embodiment]
(Configuration)
x F* I? &?'£ I- Irfr x 51 " x '<£. ~ '£ ts 3L ^ 3L fa - x jfL -
First, the configuration of the multi-tenant
system of the exemplary embodiments will be described.
FIG. 3 is a diagram showing the whole configuration of
a multi-tenant system according to a first exemplary
5 embodiment of the present invention. The multi-tenant
system of the present exemplary embodiment includes an
open flow controller (OFC) 1, a first open flow switch
(OFS). 2-1, a second open flow switch (OFS) 2-2, a
server apparatus 3, a bridge 5 and a SAN 6. The
10 server apparatus 3 includes a virtual machine (VM) 4-1
and a virtual machine (VM) 4-2. The configuration
example of FIG. 3 has a simple configuration for the
description but apparatuses of an optional number may
be connected with the multi-tenant system. Also, in
15 the present exemplary embodiment, the description is
made as the resources allocated to the user are the
virtual machines (VM) which operate on the server
apparatus but may be physical terminals such as
personal computers.
20 Hereinafter, the first open flow switch (OFS)
2-1 and the second open flow switch (OFS) 2-2 are
intensively called the open flow switches (OFS) 2, in
case of not distinguishing between them. Also, the
virtual machine (VM) 4-1 and the virtual machine (VM)
25 4-2 are intensively called the virtual machines (VM) 4
in case 'of not distinguishing between them.
FIG. 4 is.a block diagram showing the open
flow controller (OFC) 1 in the multi-tenant system
according to the first exemplary embodiment of the
30 present invention. The open flow controller (OFC) 1
of the present exemplary embodiment includes a route
calculating section 11 and a flow entry setting
section 12. The route calculating section 11 ~*
calculates a transfer route of the open flow switches
35 (OFS) 2 based on the received packet subjected to
packet-in. The route calculating section 11 specifies
- 9
a destination equipment, a transmission source
equipment and the switches connected with the
respective equipments based on a destination address
and a source address in the packet, and calculates the
5 route between the equipments as the transfer route
based on topology data of the system (not shown). The
flow entry setting section 12 sets flow entries which
are necessary for transfer control of the packets, to
the open flow switches (OFS) 2 on the transfer route
10 calculated by the route calculating section 11.
FIG. 5 is a block diagram of the open flow
switch (OFS) 2 in the multi-tenant system according to
the first exemplary embodiment of the present
invention. The open flow switch (OFS) 2 of the
15 present exemplary embodiment includes a transfer
processing section 21, a tunneling determining section
22, a processing section 23, an address translating
section 24 and a storage section 30. A flow table 25,
a MAC address table 26, a tenant management table 27,
20 a gateway management table 28 and an address
translation table 29 are stored in the storage section
30.
The transfer processing section 21 transfers.
packets to a destination which is determined based on
25 the tables stored in the storage section 30. The
tunneling determining section 22 determines whether or
not the packet received by the open flow switch (OFS)
2 has been encapsulated. Also, the tunneling
determining section 22 determines whether or not the
30 packet received by the open flow switch (OFS) 2 has to
be encapsulated. The processing section 23 carries
out the encapsulation of the packet received by the
open flow switch (OFS) 2 and decapsulation of the
encapsulated packet. The address translating section
35 24 translates a port number and an IP address which is
contained in the packet, based on the .address
JLTJ, * e R_ FT JL !OA_ CE A qj. J Q • JLS'
- 10 -
translation table 29. Note that the translation may
be carried out to only either of the IP address or the
port number.
Next, the tables stored in the storage
5 section 30 will be described.
FIG. 6 is an example of the flow table 25
according to the first exemplary embodiment of the
present invention. The flow table 25 manages the flow
entry which contains an operation rule to a received
10 packet which is defined based on a "rule", an "action"
and "statistics". Data used to identify the packet is
set to the "rule". For example, it is data as a
combination of a VLAN ID and a packet transmission
source IP address and so on. The "action" defines how
15 the open flow switch (OFS) 2 should process the packet
conforming to the data contained in the "rule". For
example, a process of transmitting to a node having a
predetermined IP address from a predetermined port of
the open flow switch (OFS) 2 and a process of
20 discarding a packet and so on are defined in the
"action". For example, by combining the "rule" and
the "action", the operation of discarding the packet
can be set to the open flow switch (OFS) 2 when the
protocol number is "1" (ICMP). The "statistics" are
25 statistic data every flow entry. For example, it is
the number of transfer packets and the number of
transfer octets.
FIG. 7 is an example of the MAC address table
26 according to the first exemplary embodiment of the
30 present invention. The MAC address table 26 manages a
MAC address of an equipment which is connected with
the open flow switch (OFS) 2, and a switch port number
of the open flow switch (OFS) ~2""wTfTh"T^n315h_-t"n~e ~- ~~
equipment having the MAC address is connected. Note
35 that the MAC address may be a virtual MAC address
- 11 -
which is allocated to the virtual machine (VM) and so
on .
FIG. 8 is an example of the tenant management
table 27 according to the first exemplary embodiment
5 of the present invention. The tenant management table
27 manages a MAC address of the equipment which is
connected with the open flow switch (OFS) 2 and tenant
identification (ID) data of a tenant system to which
the equipment having the MAC address belongs. Note
10 that the MAC address may be a virtual MAC address
which is allocated to a virtual machine (VM). Also,
it is not necessary for the virtual machine (VM) to
have the tenant identification data and a tenant to
which the virtual machine (VM) belongs may be managed
15 by the open flow switch (OFS) giving the tenant
identification data to the virtual machine (VM). For
example the tenant identification data can be managed
in a layer of Hyper-Visor. In this case, the open
flow switch (OFS) 2 which receives the packet
20 transmitted from the virtual machine (VM) inquires a
tenant to which the virtual machine (VM) belongs, to
the Hyper-Visor and adds the tenant identification
data to the packet.
FIG. 9 is an example of the gateway
25 management table 28 according to the first exemplary
embodiment of the present invention: The gateway
management table 28 manages an IP address of a gateway
for every gateway ID. The gateway ID is an identifier
which identifies the gateway uniquely, and is
30 allocated to be incremented from an optional number
expressed with "1" and "0".
FIG. 10 is an example of the address
translation table 29 according to the first exemplary^
embodiment of the present invention. The address
35 translation table.29 manages a destination IP address,
a destination port number, a destination tenant, an
- 12 -
allocated IP address, an allocated port number, a
source gateway and a source virtual machine (VM). The
destination IP address and the destination port number
indicate an IP address and a port number of an
5 equipment which is connected with the multi-tenant
system and whose tenant identification data cannot be
recognized (hereinafter, to be referred to as an
"identification impossible unit") . The destination
tenant is a tenant which is connected with the multi-
10 tenant system and which is allocated to the
identifying impossible unit. By allocating a
plurality of tenants to one identification impossible
unit, a plurality of tenant systems may share the
identification impossible unit. The allocated IP
15 address and the allocated port number are a source IP
address .and a source port number set for the
identification impossible unit when a' packet is
transmitted to the identification impossible unit from
the open flow switch (OFS) 2. The source gateway is
20 data used to identify the source open flow switch
(OFS) 2 of the packet having header data to be
rewritten based on the address translation table 29.
The source virtual machine (VM) is data used to
identify the source virtual machine (VM) 4 of the
25 packet having the header data to be rewritten, based
on the address translation table 29. Note that the
address translation table 29 may manage only the
destination IP address and the allocated IP address
when the destination port number and the allocated
30 port number are not used.
(Operation)
Next, a packet transferring method in themulti-
tenant system of the pr•ers"&n"1T*"e-x^"m^'-l-"ar"y^^-~ ----"- — - --—;-•
embodiment will be described.
35 First,, the packet transferring method (outbound)
will be described. FIG. 11 is a flow chart of
- 13 -
the packet transferring method (out-bound) in the .
multi-tenant system according to the first exemplary
embodiment of the present invention. FIG. 12 is a
diagram schematically showing a packet on the out-
5 bound route (the virtual machine (VM) 4-1 -> SAN 6) in
the packet transferring method (out-bound) of FIG. 11.
The operation when the first open flow switch (OFS) 2-
1 receives a packet transmitted from the virtual
machine (VM) 4-1 to the SAN 6 as the identification
10 impossible unit for the first time will be described
below.
(Step SI)
The virtual machine (VM) 4-1 belonging to the
tenant A system transmits a packet for SAN- 6 as the
15 identification impossible unit. Note that in FIG. 12,
the packet, (src: virtual machine (VM) 4-1, dst: SAN 6)
which is transmitted to the first open flow switch
(OFS) 2-1 from the server apparatus 3 corresponds to
the above packet. Here, "src: VM 4-1" indicates that
20 the source IP address is an IP address which is
allocated to the virtual machine (VM) 4-1. In the
same way, "dst: SAN 6" shows that the transmission
destination IP address is an IP address allocated to
the SAN 6.
25 (Step S2)
The first open flow switch (OFS) 2-1 receives
the packet transmitted at the step SI. The first open
flow switch (OFS) 2-1 compares a MAC address
corresponding to the destination IP address of the
30 packet and the MAC address table 26. The first open
flow switch (OFS) 2-1 determines that an equipment
corresponding to the MAC address is connected with
another open flow"switch (OFS )""2" (or"7"*t"h"e""u'ni"fT""""
corresponding to the MAC address is not connected with
35 the first open flow switch (OFS) 2-1), when the MAC
address is not managed in the MAC address table 26.
, _ A^^gJ5jLJ!iJL-,^jLrJL^j: ^1 SZlM^kJSL
- 14 -
In this case, the first opei\ flow switch (OFS) 2-1
determines that it is necessary to encapsulate the
packet. In this example, the first open flow switch
(OFS) 2-1 refers to the packet to determine that the
5 equipment corresponding to the destination IP address
of the received packet (the SAN 6 in this example) is
not connected with the first open flow switch (OFS) 2-
1 .
(Step S3)
10 The first open flow switch (OFS) 2-1 carries
out packet-in of the packet into the open flow
controller (OFC) 1, in order to inquire the transfer
route.of the received packet and a destination IP
address in encapsulating the packet to the open flow
15 controller (OFC) 1. In this example, the first open
flow switch (OFS) 2-1 determines whether or not the
encapsulation should be carried out, based on the MAC
address table 26, at the step S2, and determines
whether or not it is necessary to inquire data
20 necessary for the encapsulation to the open flow
controller (OFC) 1 (Step S3). Note that the first
open flow switch (OFS) 2-1 may determine that the
inquiry to the open flow controller (OFC) 1 is
necessary, without referring to the MAC address table
25 26 when the flow table 25 does not have any flow entry
used to transfer the received packet.
(Step S4)
The open flow controller (OFC) 1 specifiesthat
the open flow switch (OFS) 2 connected with the
30 SAN 6 as destination of the packet subjected to the
packet-in is the second open flow switch (OFS) 2-2.
The open flow controller (OFC) 1 calculates the outbound
route used to transmit the packe"t"™iTi""p'ac1c~eT"'*_l7b~
the second open flow switch (OFS) 2-2.
35 (Step S5)
„!"' %A t.C..k...f!G-.&.. &i A ISI4, l& VXI
- 15 -
The open flow controller (OFC) 1 sets a flow
entry used to transfer the packet to each of the
switches on the out-bound route calculated at the step
S4. Note that in FIG. 12, only the first open flow
5 switch (OFS) 2-1 and the second open flow switch (OFS)
2-2 are shown to simplify the description. However, a
plurality of open flow switches (OFS) 2 may exist
between the first open flow switch (OFS) 2-1 and the
second open flow switch (OFS) 2-2. In such a case,
10 the flow entry is set to each of the open flow
switches (OFS) 2 between the first open flow switch
(OFS) 2-1 and the second open flow switch (OFS) 2-2. •
Also, the open flow controller (OFC) 1 sets
the flow entry which prescribes an operation of
15 encapsulating the packet by adding addition data, to
the first open flow switch (OFS) 2-1 as a packet-in
source.
Also, the open flow controller (OFC) 1 sets
the flow entry which prescribes an operation of
20 removing the addition data (decapsulation) to the
second open flow switch (OFS) 2-2 as a termination
edge connected with the identification impossible unit
(Step S6)
The open flow controller (OFC) 1 carries out
25 packet-out of the packet subject to the packet-in at
the step S3, to the first open flow switch (OFS) 2-1.
(Step S7)
The first open flow switch (OFS) 2-1 produces
an encapsulated packet by encapsulating the packet
30 subjected to the packet-in (the packet received at the
step S2) based on the flow entry set by the open flow
controller (OFC) 1 by using the addition data. For
example, the addition data contains aiTL3 header "~""'-
(source IP address: open flow switch (OFS) 2-1,
35 destination IP address: open flow switch (OFS) 2-2)
and the tenant identification data (tenant A ) . Here,
,.£"%£ «!&£.. «r£.A_ O F * ~ *--<£ .2© 1. ^
- 16 -
the "source IP address: open flow switch (OFS) 2-1"
means that the source IP address of the encapsulated
packet is the IP address of the first open flow switch
(OFS) 2-1. In the same way, the "destination IP
5 address: open flow switch (OFS) 2-2" means that the
destination IP address of the encapsulated packet is
the IP address of the second open flow switch (OFS) 2-
2 .
(Step S8)
10 The second open flow switch (OFS) 2-2.
receives the encapsulated packet from the first open
flow switch (OFS) 2-1. The second open flow switch
(OFS) 2-2 extracts the tenant identification data, and
the IP addresses of the first open flow switch (OFS)
15 2-1 as a source gateway from the addition data which
is contained in the encapsulated packet, and extracts
the IP address of the SAN 6 as a destination equipment
and the IP address of the source virtual machine (VM)
4-1 from the header data of the packet other than the
20 addition data, and records them in the address
translation table 29. Also, the second open flow
switch (OFS) 2-2 allocates DUM to the source IP
address (allocated IP address of the address
translation table 29) used for the transfer of data
25 contained in the encapsulated packet. In this example,
the record of the address translation table'29
corresponding to the encapsulated packet is set to be
shown in FIG. 12.
(Step S9)
30 The second open flow switch (OFS) 2-2 removes
the addition data given at the step S7 based on the
flow entry set at the step S5, i.e. decapsulates the
encapsulated packet. - ;—• --- —: —
(Step S10)
35 The second open flow switch (OFS) 2-2
translates the transmission source IP address into DUM
jueriM:—tsLtrjE-joLi saLx. - • * . £ , . " ^-aLA^gt__jLs3Li^g^
- 17 -
based on the address translation table 29 of FIG. 12.
Note that in the address translation table 29 of the
second open flow- switch (OFS) 2-2 shown in FIG. 12,
the destination port number and the allocated port
5 number are not used. In this way, the packet may be
rewritten by using only the destination IP address,
the destination tenant, the allocated IP address and
the source gateway. That is, the second open flow
switch (OFS) 2-2 rewrites the packet by using a NAT
10 function or an IP masquerade function. For example,
when the NAT function is used, ,the second open flow
switch (OFS) 2-2 outputs a unique IP address as the
DUM for every source virtual machine (VM) from the
plurality of IP addresses managed by the second open
15 flow switch (OFS) 2-2. Or, when the IP masquerade
function is used, the second open flow switch (OFS) 2-
2 outputs a unique set of an IP address and a port
number as the DUM for every source virtual machine
(VM) from the - plurality of IP addresses and the
20 plurality of port numbers managed by the second open
flow switch (OFS) 2-2.
(Step Sll)
The second open flow switch (OFS) 2-2
transmits for the SAN 6, the packet that the source IP
25 address has been rewritten at the step S9.
(Step S12)
The SAN 6 receives the rewritten packet
through the bridge 5. Note that the SAN 6 may be
connected with the second open f,low switch (OFS) 2-2
30 without going through the bridge 5.
This is the packet transferring method (outbound)
of the present exemplary embodiment. Note that
the above-mentioned description is made for the
operation when the first open flow switch (OFS) 2-1
35 receives the packet transmitted from the virtual
machine (VM) 4-1 to the SAN 6 for the first time (the
- 18 -
first packet). The second and subsequent packets
received by the first open flow switch (OFS) are
transferred by the open flow switches (OFS) 2 on the
transfer route based on the flow entries set at the
5 step S5.
Next, a packet transferring method (in-bound)
will be described. FIG. 13 is a flow chart of the '
packet transferring method (in-bound) in the multitenant
system according to the first exemplary
10 embodiment of the present invention. FIG. 14 is a
diagram schematically showing a packet on the in-bound
route (SAN 6 -• virtual machine (VM) 4-1) in the packet
transferring method (in-bound) of FIG. 13. The
operation when the second open flow switch (OFS) 2-2
15 receives the packet destined to the virtual machine
(VM) 4-1 from the SAN 6 to for the first time will be
described below.
(Step S21)
The SAN 6 which is the identification
20 impossible unit transmits a reply packet to the packet
received at the step S12 to the virtual machine (VM)
4-1. Note that the packet (src: SAN 6, dst: DUM)
which is transmitted from the SAN 6 toward the second
open flow switch (OFS) 2-2 through the bridge 5
25 corresponds to the packet in FIG. 14.
(Step S22)
Because the destination IP address of the
received packet is DUM, the second open flow switch
(OFS) 2-2 refers to the address translation table 29
30 to translate the destination IP address. In this case,
the second open flow switch (OFS) 2-2 refers to the
source virtual machine (VM) of the address translation
table 29 o f F I G^r"n~t"o_tran-s-ra"te the destination IP- •--
address from the DUM to the virtual machine (VM) 4-1.
35 (Step S23)
JUg_^_Jl£J- j^jL-_-gL!.JzJt^l;.-- '£. &k-X • $L IMJr 1. W
- 19 -
The second open flow switch (OFS) 2-2
encapsulates the packet having the address translated
at the step S22. In detail, the second open flow
switch (OFS) 2-2, encapsulates the packet transmitted
5 at the step S21 by adding the addition data, to
produce an encapsulated packet. In this case, the
second open flow switch (OFS) 2-2 refers to the source
gateway of the address translation table 29 to specify
that the source gateway is the first open flow switch
10 (OFS) 2-1. The second open flow switch (OFS) 2-2
refers to the destination tenant of the address
translation table 29 to specify the tenant data to be
included in the addition data. The second open flow
switch (OFS) 2-2 makes the addition data include an L3
15 header (source IP address: open flow switch (OFS) 2-2,
destination IP address: open flow switch (OFS) 2-1)
and tenant identification data (tenant A ).
(Step S24)
Because the encapsulated packet is the first
20 packet, the first open flow switch (OFS) 2-1 carries
out packet-in of the encapsulated packet into the open
flow controller (OFC) 1.
(Step S25)
The open flow controller (OFC) 1 calculates
25 the in-bound route based on the encapsulated packet
subjected to the packet-in. In this case, the open
flow controller (OFC) 1 specifies a source equipment
and a destination virtual machine (VM) from the source
IP address and the destination IP address of the
30 packet obtained by decapsulating the encapsulated
packet subjected to the packet-in, and specifies an
end edge (gateway) connected with the destination
virtual machine (VM) by using topology da't"a~"(no"t'
shown) and calculates the in-bound route.
35 (Step S26)
31^^SSxIa3^IZI2m^3Srii
- 20 -
The open flow controller (OFC) 1 sets the
flow entries for transferring the encapsulated packet
to the switches on the in-bound route calculated at
the step S25. Note that only the first open flow
5 switch (OFS) 2-1 and the second open flow switch (OFS).
2-2 are shown in FIG. 14 to simplify the description,
but a plurality of open flow switches (OFS) 2 may
exist between the first open flow switch (OFS) 2-1 and
the second open flow switch (OFS) 2-2. In such a case,
10 the flow entries are respectively set to the open flow
switches (OFS) 2 between the first open flow switch
(OFS) 2-1 and the second open flow switch (OFS) 2-2.
Also, the open flow controller (OFC) 1 sets
the flow entry which prescribes the decapsulating
15 operation of removing the addition data to the last
open flow switch (OFS) 2 on the in-bound route (the
first open flow switch (OFS) 2-1 connected with the
virtual machine (VM) 4-1).
(Step S27).
20 The open flow controller (OFC) 1 carries out
packet-out of the encapsulated packet subjected to the
packet-in at the step S24, to the second open flow
switch (OFS) 2-2. '•
(Step S28)
25 The second open flow switch (OFS) 2-2
transmits the encapsulated packet to the first open
flow switch (OFS) 2-1 according to the flow entry set
at the step S26.
(Step S29)
30 The first open flow switch (OFS) 2-1 removes
the addition data added at the step S23 from the
encapsulated packet based on the flow entry set at the
step S26. Also, the first open, flow- switch --(-OFS-)— : 2 — 1 ——
refers to the tenant data contained in the addition
35 data to check the transmission destination of the
>rs . eii.8.I. , t i " l £ ~ '£S 1_4
- 21 -
decapsulated packet, i.e. the packet that the addition
data has been removed.
(Step S30)
The first open flow switch (OFS) 2-1
5 transmits the decapsulated packet to the virtual
machine (VM) 4-1. The virtual machine (VM) 4-1
receives the decapsulated packet.
According to the present exemplary embodiment,
the multi-tenant system is provided which can use. the
10 equipment whose tenant identification data cannot be -
recogni zed.
Next, as an example of the multi-tenant
system in which a part of the configuration of the
multi-tenant system according to the first exemplary
15 embodiment of the present invention is changed, a
second exemplary embodiment will be described.
[Second Exemplary Embodiment]
(Configuration)
20 First, the configuration of the multi-tenant
system of a second exemplary embodiment will be
described. Because the whole configuration of the
multi-tenant system of the present exemplary
embodiment is same as that of the multi-tenant system
25 of the first exemplary embodiment shown in FIG. 3, the
detailed description is omitted.
FIG. 15 is a block diagram of the open flow
controller (OFC) 1 in the multi-tenant system of the
second exemplary embodiment of the present invention.
30 The open flow controller- (OFC) 1 of the present
exemplary embodiment includes the route calculating
section 11, the flow entry setting section 12, a
tunneling determining section 13 and a"~'s'"forage~ ""sec'fTorT
16. The storage section 16 stores a tenant management
35 table 27 and a gateway management table 28. The route
calculating section 11 calculates the transfer route
,.r« ?tA»..(3! • x«; <£ <0\ X. .*•! _ . X O- - . X . «R
- 22 -
based on the packet subjected to the packet-in from
the open flow switch (OFS) 2. The flow entry setting
section 12 sets the flow entries which are necessary
for the transfer control of the packet, to the open
5 flow switches (OFS) 2 on the transfer route calculated
by the route calculating section 11. The tunneling
determining section 13 determines whether or not the
packet received from the open flow switch (OFS) 2 has
been encapsulated. Also, the tunneling determining
10 section 22 determines whether or not the packet
received from the open flow switch (OFS) 2 has to be
encapsulated.
FIG. 16 is a block diagram of the open flow
switch (OFS) 2 in the multi-tenant system according to
15 the second exemplary embodiment of the present
invention. The open flow switch (OFS) 2 of the
present exemplary embodiment includes the transfer
processing section 21, the processing section 23, the
address translating section 24 and a storage section
20 30. The storage section 30 stores the flow table 25,
the MAC address table 26 and the address translation
table 29.
Because the transfer processing section 21
transfers the packet to a destination determined based
25 on the respective tables which are stored-in the
storage section 30. The processing section 23 carries
out the encapsulation of the packet.received from the
open flow switch (OFS) 2 and decapsulation of the
encapsulated packet. The address translating section
30 24 translates an IP address and a port number, which
are contained in the packet, based on the address
translation table 29. Note that the translation may
be only either of the IP address" or" tKe~ |Tor"1f"n"umb~er.'
The configuration of each of the tables which
35 are stored in the storage section 16 of the open flow
controller (OFC) 1 and the storage section 30 of the
"UPSt G£iMl, . _SI."-I2_"_£:§: I-^. -i.S-Lil-. - •- ,--,—
- 23 -
open flow switch (OFS) 2 is the same as' that of each
table in the first exemplary embodiment, and therefore,
a detailed description is omitted.
(Operation)
5 Next, a packet transferring method in the
multi-tenant system of the present exemplary
embodiment will be described. Because the packet
transferring method (in-bound) in the present
exemplary embodiment is same as that of the first
10 exemplary embodiment, only a packet transferring
method (out-bound) will be described.
First, the packet transferring method (outbound)
will be. described. FIG. 17 is a flow chart of
the packet transferring method (out-bound) in the
15 multi-tenant system according to the second exemplary
embodiment of the present invention. Fig. 17 is same
as FIG. 12 with respect to the diagram schematically
showing the packet on the out-bound route (virtual
machine (VM) 4-1 - SAN 6).
20 (Step S31)
The virtual machine (VM) 4-1 which belongs to
the tenant system transmits a packet to the SAN 6
which is an identification impossible unit. Note that
in FIG. 12, the packet (src: virtual machine (VM) 4-1,
25 dst: SAN 6) which is transmitted to the first open
flow switch (OFS) 2-1 from the server apparatus 3
corresponds to the packet. Here, "src: virtual
machine (VM) 4-1" means that the source IP address is
an IP address which is allocated to the virtual
30 machine (VM) 4-1. In the same way, "dst: SAN 6" means
that the destination IP address is an IP address which
is allocated to the SAN 6.
(Step S32) . --- . _.._„..__„_ _
The first open flow switch (OFS) 2-1 receives
35 the packet transmitted.at the step S31.
(Step S33)
T^ X. Q: - X
- 24 -
Because there is not a flow entry which
prescribes an operation.of receiving the packet
(because the first, packet is received), the first open
flow switch (OFS) 2-1 carries out the packet-in of the
packet into the open flow controller (OFC) 1.
(Step S34)
The open flow controller (OFC) 1 determines
whether or not the packet has to be encapsulated based
on a destination MAC address of the packet subjected
to the packet-in at the step S33. When the
destination MAC address is not one of MAC addresses
managed by the first open flow switch (OFS) 2-1, t'he
open flow controller (OFC) 1 determines to have to
transmit the packet through other open flow switches
(OFS) 2. When the open flow controller (OFC) 1
determines to have to transmit the packet through
other open flow switches (OFS) 2, the open flow
controller (OFC) 1 determines that the packet has to
be encapsulated.
(Step S35)
The open flow controller (OFC) 1 specifies
that the open flow switch (OFS) 2 connected with the
SAN 6 as the destination of the packet subjected to
the packet-in is the second open flow switch (OFS) 2-2,
based on topology data (not shown). The open flow
controller (OFC) 1 calculates the out-bound route used
to transmit the packet, subjected to the packet-in, to
the second open flow switch (OFS) 2-2.
(Step S36)
The open' flow controller (OFC) 1 sets the
flow entries used to transfer the packet to the
switches on the out-bound route calculated at the step
S35. Note that only the first open flow .switcii-(Of-S-)
2-1 and the second open flow switch-(OFS) 2-2 are
shown in FIG. 12 in order to simplify the description.
However, a plurality of open flow switches (OFS) 2 may
25 -
exist between the first open flow switch (OFS) 2-1 and
the second open flow switch (OFS) 2-2. In this case,
flow entries are set to the open flow switches (OFS) 2
between the first open flow switch (OFS) 2-1 and the
5 second open flow switch (OFS) 2-2.
Also, the open flow controller (OFC) 1 sets a
flow entry which prescribes an operation of
encapsulating the packet by adding the addition data
to the first open flow switch (OFS) 2-1 as a packet-in
1 0 source.
Also, the open flow controller (OFC) 1 sets a
flow entry which prescribes a decapsulating operation
of removing the addition data to the second open flow
switch (OFS) 2-2 as a termination edge connected with
15 the identification impossible unit.
(Step S37)
The open flow controller (OFC) 1 carries out
• the packet-out of the packet subjected to the packetin
at the step S32, to the first open flow switch
20 (OFS) 2-1. When the open flow controller (OFC) 1
determines that the first open flow switch (OFS) 2-1
has to encapsulate the packet subjected- to the packetin
at the step S34 and has to transfer the
encapsulated packet, the open flow controller (OFC) 1
25 make a Packet-Out message include the packet and the
tenant identification data of the packet.
(Step S38)'
The first open flow switch (OFS) 2-1
encapsulates the packet subjected to the packet-in
30 (the packet received at the step S2) based on the flow
entry set by the open flow controller (OFC) 1 by
adding the addition data to produce an encapsulated
packet. For example, the addition data contains an L3
• header (source IP address: open flow switch (OFS) 2-1,
35 destination IP address: open flow switch (OFS) 2-2)
and the tenant identification data (tenant A ) . Here,
%J. %J C L ft S,. -!• -Jt £.Cti
~~^|-~£^ *• -a. c"n
- 26 -
the "source IP address: open flow switch (OFS) 2-1"
means that the source IP address of the encapsulated
packet is an IP address of the first open flow switch
(OFS) 2-1. In the same way, the "destination IP
5 address: open flow switch (OFS) 2-2" means that the
destination IP address of the encapsulated packet is
an IP address of the second open flow switch (OFS) 2-2.
(Step S39)
The second open flow switch (OFS) 2-2
10 receives the encapsulated packet from the first open
flow switch (OFS) 2-1.
(Step S40)
The second open flow switch (OFS) 2-2 removes
the addition data added at the step S37 based on the
15 flow entry set to at the step S36.
(Step S41)
The second open flow switch (OFS) 2-2
translates the source IP address into DUM based on the
address translation table 29 of FIG. 12, like the
20 first exemplary embodiment.
(Step S42)
The second open flow switch (OFS) 2-2
transmits to the SAN 6, the packet in which the source
IP address has been rewritten at the step S40.
25 (Step S43)
The SAN 6 receives the rewritten packet
through the bridge 5. Note that the SAN 6 may.be
connected with the second open flow switch (OFS) 2-2
without going through the bridge 5.
30 According to the present exemplary embodiment,
the multi-tenant system is provided which can use
existing identification impossible unit. Note that
the description of the exemplary embodiments of the
present invention have been made by using the SAN as
35 an example of the identification impossible unit.
However, the identification impossible unit is not
- 27 -
limited to the SAN and the present invention may be
applied to other units (equipments).
In the above, the exemplary embodiments of
the present invention have been described with
5 reference to the attached drawings. However, the
present invention is not limited to the abovementioned
exemplary embodiments and can be
appropriately modified by a skilled person in the art
in a range which does not deviate from the scope of
10 the present invention.
Note that this application claims a priority
on convention based on Japanese Patent Application No,
JP 2012-111881. The disclosure thereof is
incorporated herein by reference.
15

CLAIMS
1. A multi-tenant system comprising:
a server apparatus on which a virtual machine
with tenant identification data given operates;
5 a plurality of switches, each of which
comprises a processing section which processes a
packet based on a flow entry set to said switch; and
a controller configured to set the flow entry
to each of said plurality of switches,
10 wherein said plurality of switches includes a
first switch connected with said server apparatus and
a second switch connected with an equipment whose
tenant identification data cannot be recognized,
wherein said controller comprises a flow
15 entry setting section configured to set to said first
switch, the flow entry which prescribes an operation
of encapsulating a packet which is transmitted from
said virtual machine on said server apparatus, by
adding addition data which includes the tenant
20 identification data, and set to said second switch,
the flow entry which prescribes a decapsulating
operation of removing the addition data from the
encapsulated packet, and
wherein said second switch comprises:
25 an address translating section configured to
translate a source address of the decapsulated packet
by said processing section, into an IP address managed
by said second switch; and
a transfer processing section configured to
30 transfer the packet, whose source address has been
translated, for said equipment.
2. The multi-tenant system according to claim 1,
wherein said controller carries out packet-out of a
35 packet-out message which includes the packet and the
tenant identification data of the packet, and
- 29 -
wherein said processing section of said first
switch encapsulates the packet based on the packet-out
message.
5 3. The multi-tenant system according to claim 1
or 2 , wherein said second switch stores in said
address translation table, an IP address of said first
switch, a IP address of said virtual machine as a
transmission source and the tenant identification data
10 which are contained in the encapsulated packet when
. the encapsulated packet is received,
wherein said flow entry setting section of
said controller sets to said second switch, the. flow
entry which prescribes the operation of encapsulating
15 the packet transmitted from said equipment by adding
the addition data containing the tenant identification
data, and sets to said first switch, the flow entry
which prescribes the decapsulating operation of
removing the addition data from the encapsulated
2 0 packet,
wherein said transfer processing section of
said second switch receives a reply packet transmitted
to said virtual machine from said equipment,
wherein said address translating section of
25 said second switch translates a destination IP address
of the reply packet into the IP address of said
virtual machine based on said address translation
table,
wherein said the processing section of said
30 second' switch encapsulates the reply packet subjected
to the translation of the source IP address, to the
encapsulated reply packet in which the addition data
containing the tenant i d e n't if 1 cat i"o n"~d"a t a""'©!' 'sarct'J~~~
virtual machine has been added, wherein an IP address
35 of said fir.st switch of said address translation table
xjgsjrj: Olfe-l-JSLx. ELI, "* JL£, ~ '£,. q' 1-JS3L JL. Q - 3. 5
- 30 -
is set as a destination IP address of an L3 header of
the addition data,
wherein said processing section of said first
switch removes the addition data from the encapsulated
5 reply packet, and
wherein said transfer processing section of
said first switch transmits the reply packet in which
the addition data has been removed, to said virtual
machine .
10
4. The multi-tenant system according to any of
claims 1 to 3, wherein said address translating
section of said second switch translates a source port
number of the packet into a port number managed by
15 said second switch and translates a destination port
number of the reply packet into a source port number
before the translation.
5. The controller which is used in the multi-
20 tenant system according to any of claims 1 to 4.
6. The switch which is used in the multi-tenant .
system according to any of claims 1 to 4.
25 7. A packet transferring method in a multitenant
system, comprising:
transmitting a packet from a virtual machine
(VM) to an equipment whose tenant identification data
cannot be recognized;
30 carrying out packet-in of the packet into a
controller by a first switch;
calculating an out-bound route of the packet
by said controller;
setting flow entries for packet transfer into
35 switches on the out-bound route by said controller;
- 31 -
wherein said setting comprises setting to the
first switch, a flow entry which prescribes an
operation of encapsulating the packet by adding
addition data which contains tenant identification
5 data of said virtual machine (VM) to the packets, and
setting in said second switch, a flow entry which
prescribes a decapsulating operation of removing the
addition data from the encapsulated packet;
encapsulating, by said first, switch, the
10 packet based on the set flow entry by adding the
addition data which contains the tenant identification
data of said virtual machine (VM);
receiving the encapsulated packet by said
second switch;
15 removing by said second switch, the addition
data from the encapsulated packet based on the flow
entry set in said second switch;
translating a transmission IP address of the
decapsulated packet from which the addition data has
20 been removed into an IP address managed by said second
switch based on an address translation table; and
transmitting the translated packet to said
equipment by said second switch.
25 8. The packet transferring method according to
claim 7, further comprising:
carrying out packet-out of a packet-out
message which contains the packet and the tenant
identification data of the said packet by said
30 controller.
9. The packet transferring method according to
claim1 7 or 8, further comprising:
recording in the address translation table by
35 said second switch which has received the encapsulated
packet, an IP address of said first switch, an IP
- 32 -
address of a source virtual machine (VM), and the
tenant identification data which are contained in the
encapsulated packet;
transmitting a reply packet from said
5 equipment to said virtual machine (VM);
translating a destination IP address into the
IP address of said source virtual machine (VM) based
on the address translation table by said second
switch;
10 encapsulating the reply packet by adding the
addition data which contains the tenant identification
data of said virtual machine (VM) by said second
switch to produce an encapsulated reply packet;
setting the IP address of said first switch
15 to the address translation table as a destination IP
address of an L3 header in the addition data;
carrying out packet-in of the encapsulated
reply packet into said controller by said second
switch;
20 calculating an in-bound route of the
encapsulated reply packet by said controller;
setting to the switches on the in-bound route,
the flow entries for transferring the encapsulated
reply packet by said controller;
25 wherein setting a flow entry which prescribes
an operation of removing the addition data, to said
first switch;
carrying out packet-out of the encapsulated
reply packet to said second switch by said controller;
30 transmitting the encapsulated reply packet to
said first switch from said second switch;
receiving the encapsulated reply packet by
said first switch;
removing the addition data based on the flow
35 entry by said first switch; and
- 33 -
transmitting the reply packet having the
addition data removed, to said virtual machine (VM) by
said first switch.
10
10. The packet transferring method according to
any of claims 7 to 9, wherein said encapsulating the
packet comprises encapsulating the packet in which a
source port number of the packet is translated into a
port number managed by said second switch; and
wherein said encapsulating the reply packet
comprises encapsulating the reply packet in which a
destination port number of the reply packet into a
source port number before the translation.