FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENTS RULES, 2003
COMPLETE SPECIFICATION (See section 10, rule 13)
“INFORMATION PROCESSING DEVICE AND INFORMATION PROCESSING METHOD”
SONY CORPORATION, of 1-7-1, Konan, Minato-ku, Tokyo 108-0075, Japan
The following specification particularly describes the invention and the manner in which it is to be performed.
In this case, the user of the agricultural field B is capable of easily applying the ecosystem object indicating the wormwood to the wormwood of the agricultural field B. [0575]
In addition, for example, in a case where the teacher user, for example, instructs the student user performing an operation in the agricultural field B, to plant the wormwood as the advice of the vegetation design, in the ecosystem objects of the agricultural field A, the ecosystem object indicating the wormwood can be copied to a position in the agricultural field B where the wormwood is planned to be planted. In this case, the teacher user is capable of easily advising the student user against the planting of the wormwood. [0576]
Here, for example, as described above, in a case where the teacher user copies the ecosystem object in order to advise the student data, in the server 13, a task object indicating a task to be performed with respect to the ecosystem constituent indicated by the ecosystem object of the copy target can be added (registered). [0577]
That is, for example, in a case where the ecosystem object indicating the wormwood is not applied to the agricultural field B, in the server 13, the editing unit 75 specifies a task to be performed with respect to the wormwood indicated by the ecosystem object copied to the agricultural field B (hereinafter, also referred to as a copy object) when the ecosystem object indicating the wormwood of the agricultural field A is copied to the agricultural field B. [0578]
Then, the editing unit 75 generates the task object

indicating the task to be performed with respect to the wormwood (allows the generation unit 73 to generate the task object), and registers the task object in the storage 63. [0579]
Further, in the server 13, the AR/VR display control unit 74 displays the task object registered in the storage 63 on the terminal 12. [0580]
Accordingly, in the terminal 12, the task object indicating the task to be performed with respect to the wormwood is displayed to be close to an ecosystem object as the copy object indicating the wormwood of the agricultural field B. [0581]
In Fig. 55, a task object indicating the planting of the wormwood is displayed as the task to be performed with respect to the wormwood. [0582]
As described above, the ecosystem object as the copy object indicating the wormwood and the task object indicating the planting, which are applied to the agricultural field B are capable of performing the AR display in the terminal 12 of the user of the agricultural field B when the user of the agricultural field B actually performs an operation in the agricultural field B. [0583]
In this case, the user of the agricultural field B observes the ecosystem object as the copy object indicating the wormwood and the task object indicating the planting, and thus, is capable of easily recognizing that the wormwood is planned to be planted in a position where the copy object indicating the wormwood is displayed.

[0584]
Furthermore, in Fig. 55, a case where the ecosystem object of the agricultural field A is copied to the other agricultural field B, is described, and for example, an ecosystem object in a certain position of the agricultural field A can be copied to the other position of the agricultural field A, as the copy of the ecosystem object. [0585]
[0586]
Fig. 56 is a diagram illustrating an example of a change in a display scale of the background space in the terminal 12. [0587]
In the terminal 12, in a case where the AR display or the VR display is performed by using a captured real space or a VR space as a background space, the display scale of the background space (an image of the background space) can be changed according to the operation or the like of the user. [0588]
Fig. 56 illustrates an example of a background space Ba in which a reduction scale as the display scale is small, and a background space Bb in which the reduction scale is large. [0589]
In Fig. 56, the background space Ba is an image obtained by overviewing a certain agricultural field, and the background space Bb is an image obtained by enlarging a part of the background space Ba. [0590]
Fig. 57 is a diagram illustrating an example of a change in the display of the AR object according to the change in

the display scale of the background space in the terminal 12. [0591]
In Fig. 57, the AR object (the ecosystem object and the task object) is displayed in the same background space Ba or Bb as that of Fig. 56, in a superimposed manner. [0592]
Here, in a case where all AR objects applied to the agricultural field are displayed in the background space Ba, which is the image obtained by overviewing a certain agricultural field, in a superimposed manner, the number of AR objects is excessively large, and thus, there is a case where it is difficult to see the display. [0593]
Therefore, in a case where the reduction scale as the display scale is small, that is, for example, in a case where the background space Ba, which is the image obtained by overviewing a certain agricultural field, is displayed, the AR object to the agricultural field seen in the background space Ba can be displayed to be thinned out. [0594]
In addition, in a case where the reduction scale as the display scale is large, that is, for example, in a case where the background space Bb, which is the image obtained by enlarging a part of the background space Ba, is displayed, the AR object applied to the agricultural field seen in the background space Bb can be displayed without being thinned out. [0595]
As described above, in a case where the reduction scale as the display scale is small, the AR object is displayed to be thinned out, and thus, it is possible to prevent the number

of AR objects from being excessively large, and the display
from being hardly seen.
[0596]
Furthermore, the degree of the thinning of the AR object, for example, can be set according to the display scale or the number of AR objects applied to the agricultural field seen in the background space. [0597]
In addition, in the thinning of the AR object, the maximum number of AR objects displayed on one screen is set, and the thinning of the AR object can be performed such that the number of AR objects is less than or equal to the maximum number. [0598]
[0599]
Fig. 58 is a diagram illustrating an example of the VR display using each of a plurality of agricultural fields as a background space. [0600]
As described in the copy of the ecosystem object in Fig. 55, in the terminal 12, it is possible to perform the VR display by using each VR space of two agricultural fields as a background space. [0601]
Further, in the terminal 12, the number of agricultural fields is not limited to 2, but it is possible to simultaneously perform the VR display by using each VR space of three of more agricultural fields as a background space. [0602]
In the terminal 12, in a case where the VR display is

performed by using each VR space of a plurality of agricultural fields as a background space, the VR displays of each of the plurality of agricultural fields can be displayed side by side. [0603]
In Fig. 58, the background spaces Bb and Bc where each of two agricultural fields is seen, are subjected to the VR display side by side. [0604]
In the terminal 12, as described above, the VR displays of each of the plurality of agricultural fields are displayed in side by side, and for example, the VR displays each of the plurality of agricultural fields can be displayed in a superimposed manner at a predetermined permeation rate. [0605]
In the terminal 12, the VR display is performed by using each of the VR spaces of the plurality of agricultural fields as a background space, and thus, for example, it is possible to easily compare types of vegetation or the like existing in each of the plurality of agricultural fields. [0606]
Furthermore, the display scale of the background space (the VR display in which the AR object is superimposed on the background space) in which each of the plurality of agricultural fields is seen, can be changed as described in Fig. 56 and Fig. 57. [0607]
A wide agricultural field and a narrow agricultural field are displayed such that sizes on the display are coincident with each other, by changing the display scale, and thus, for example, it is possible to easily compare a vegetation distribution of the wide agricultural field with a vegetation

distribution of the narrow agricultural field. [0608]
In addition, in the case of simultaneously performing the VR display using each of the plurality of agricultural fields as a background space (hereinafter, also referred to as a plurality of agricultural field displays), agricultural fields in which positions (locations) are different from each other, and agricultural fields in which positions are the same, but time is different, can be adopted as the plurality of agricultural fields which are used as a target of the plurality of agricultural field displays. [0609]
That is, in the plurality of agricultural field displays, for example, it is possible to perform the VR display by using each of the current state of a certain agricultural field, and the previous state of the agricultural field as a background space. In addition, for example, it is possible to perform the VR display by using each of a state of a certain agricultural field at the previous time point and a state of the agricultural field at the other previous time point as a background space. [0610]
In this case, for example, it is possible to easily compare types of vegetation with each other in a certain agricultural field at different time points. [0611]
[0612]
Fig. 59 is a diagram illustrating an association example of the marker with respect to the ecosystem object. [0613]
In the server 13, the generation unit 73, for example,

generates an ecosystem object according to the operation of the terminal 12 by the user, and as necessary, associates a marker with ecosystem object. [0614]
As described in Fig. 44 or the like, in the terminal 12, the ecosystem object can be displayed (provided) in a relative position in the background space based on the marker associated with the ecosystem object. [0615]
As illustrated in Fig. 59, one marker M1 can be associated with one ecosystem object obj1. [0616]
Further, one marker can be associated with a plurality of ecosystem objects. That is, for example, as illustrated in Fig. 59, one marker M1 can be associated with three ecosystem objects obj1, obj2, and obj3 as the plurality of ecosystem objects. [0617]
In a case where the marker M1 is associated with one ecosystem object obj1, in the terminal 12, in a case where the marker M1 is detected from an image obtained by being captured with the camera 37, one ecosystem object obj1 associated with the marker M1 is displayed in a relative position based on the marker M1. [0618]
In addition, in a case where the marker M1 is associated with three ecosystem objects obj1 to obj3, in the terminal 12, in a case where the marker M1 is detected from the image obtained by being captured with the camera 37, three ecosystem objects obj1 to obj3 associated with the marker M1 are displayed in the relative position based on the marker M1.

[0619]
The relative position based on the marker M1, displaying the ecosystem objects obj1 to obj3, can be individually set for each of the ecosystem objects obj1 to obj3. [0620]
As described above, one marker can be associated with the plurality of ecosystem objects, and thus, for example, one marker can be associated with a part or all of ecosystem objects applied to one agricultural field. [0621]
Fig. 60 is a diagram illustrating an association example of the marker with respect to the ecosystem object. [0622]
A plurality of different markers can be associated with one ecosystem object. [0623]
In Fig. 60, each of markers M1 and M2 is associated with each of the ecosystem objects obj1 to obj3. [0624]
In a case where each of the markers M1 and M2 is associated with an ecosystem object obj#i (here, i = 1, 2, and 3), a relative position based on the marker M1, in which the ecosystem object obj#i is displayed, and a relative position based on the marker M2, in which the ecosystem object obj#i is displayed, are set to be in the same position, but can be set to be in different positions. [0625]
As described above, in a case where each of the markers M1 and M2 is associated with the ecosystem object obj#i, for example, the marker M1, which is one of the markers M1 and M2, can be provided in an agricultural field where an ecosystem

constituent indicated by the ecosystem object obj#i exists, and the ecosystem object obj#i can be subjected to the AR display in the agricultural field. [0626]
Further, in the marker M2, which is the other of the markers M1 and M2, for example, the teacher user provides the marker M2 in an arbitrary location, and thus, the ecosystem object obj#i can be subjected to the AR display in such an arbitrary location. [0627]
Therefore, in a distant place separated from the agricultural field in which the ecosystem constituent indicated by the ecosystem object obj#i exists, the teacher user is capable of observing the ecosystem object obj#i indicating the ecosystem constituent existing in the agricultural farm. [0628]
Accordingly, in the distant place separated from the agricultural field, the teacher user is capable of easily recognizing the vegetation planted in the agricultural field or the arrangement of the vegetation in the agricultural field, and of capable advising the student user of the agricultural field, for example, against the vegetation design. [0629]
Furthermore, two different markers can be associated with the ecosystem object, and three or more different markers can be associated with the ecosystem object. [0630]
Fig. 61 is a diagram illustrating a change in a disposition position of the marker. [0631]

As described in Fig. 44 or the like, in the terminal 12, the ecosystem object is displayed (provided) in a relative position in the background space based on the marker associated with the ecosystem object. [0632]
In Fig. 61, the ecosystem object is displayed in a position Pb which is separated from a position Pa by only a vector Va, on the basis of the position Pa of the marker. [0633]
As described above, the ecosystem object is displayed in the relative position based on the marker, and thus, in a case where the marker is moved, an absolute position in which the ecosystem object is displayed, is also moved according to the movement of the marker. [0634]
Here, for example, in the agricultural field, there is a case where the disposition position of the marker is planned to be moved after the marker is provided. [0635]
However, in a case where the marker is moved, as described above, a position in which the ecosystem object is displayed is moved according to the movement of the marker. In this case, for example, in the AR display using the real agricultural field as a background space, the ecosystem object is displayed in a position which is not relevant to the ecosystem constituent indicated by the ecosystem object. [0636]
Therefore, the marker can be moved in a position invariant mode where a (absolute) position in which the ecosystem object associated with the marker is displayed is not changed, before and after the marker is moved.

[0637]
As described above, in the marker before being moved, the ecosystem object is displayed in a position Pb separated from a position Pa of the marker by only a vector Va. [0638]
In the position invariant mode, in a case where the marker is moved from the position Pa to a position Pc by only the vector Vb, in the marker after being moved, the ecosystem object is displayed in a position separated from the position Pc after being moved, by only a vector Vc = Va - Vb. [0639]
A position Pc + Vc = (Pa + Vb) + (Va - Vb) separated from the position Pc (= Pa + Vb) of the marker after being moved by only the vector Vc (= Va - Vb) is a position Pb = Pa + Va separated from the position Pa of the marker before being moved by only the vector Va, and thus, the ecosystem object associated with the marker is displayed in the same position Pb as that before the marker is moved even after the marker is moved. [0640]
Therefore, according to the position invariant mode, for example, in the AR display using the real agricultural field as a background space, it is possible to prevent the ecosystem object from being displayed in a position not relevant to the ecosystem constituent indicated by the ecosystem object due to the movement of the marker. [0641]
Furthermore, in the position invariant mode, in the server 13, the vector Vc = Va - Vb as a position in which the ecosystem object is displayed, based on the position Pc of the marker after being moved, is obtained. Then, the ecosystem

object is displayed in the position separated from the position Pc of the marker after being moved by only the vector Vc, that is, the same position Pb as that in the case of the marker before being moved. [0642]
[0643]
Fig. 62 is a diagram illustrating a display example of an ecosystem object indicating an ecosystem constituent relevant to the entire agricultural field. [0644]
In Fig. 62, the ecosystem object is displayed by using a certain agricultural field A (a real space of the agricultural field A, a captured real space, or a VR space) as a background space. [0645]
Further, in Fig. 62, the ecosystem object indicating each of the Chinese cabbage, the wormwood, the Chinese chive, bee-hive, and a concentration of a nitric acid, a moisture content, and climate of the agricultural field A, is displayed. [0646]
In the agricultural field A, the ecosystem object of the Chinese cabbage, the wormwood, the Chinese chive, and the bee-hive is displayed in (the vicinity of) a position where the Chinese cabbage, the wormwood, the Chinese chive, and the bee-hive exist. [0647]
On the other hand, the ecosystem object indicating each of the concentration of a nitric acid, the moisture content, and the climate of the agricultural field A is an ecosystem

object indicating an ecosystem constituent relevant to the entire agricultural field A (hereinafter, also referred to the an entire object), and thus, is displayed in a fixed position on the display screen of the display unit 35 (in Fig. 62, a position on a right end of the display screen) in which the background space of the agricultural field A is displayed. [0648]
For example, the terminal 12 is a see-through HMD, and in the see-through HMD, in the background space, the ecosystem object is subjected to the AR display in a superimposed manner, by using the real space of the agricultural field A as a background space. In this case where the user, for example, inclines his head in a right direction, in a state of being mounted with the see-through HMD as the terminal 12, the background space seen on the display screen (entering the sight of the user) is moved to a range on a right side of the agricultural field A from a state before the user inclines his head. [0649]
As a result thereof, in the background space seen on the display screen after the user inclines his head, the position in which the Chinese cabbage, the wormwood, the Chinese chive, and the bee-hive exist, is moved to a left side from the state before the user inclines his head. For this reason, the position in which the ecosystem object indicating each of the Chinese cabbage, the wormwood, the Chinese chive, and the bee-hive is displayed is also moved to the left side. [0650]
On the other hand, the entire object indicating each of the concentration of a nitric acid, the moisture content, and the climate of the agricultural field A is displayed in

the fixed position on the display screen, and thus, is displayed in the same position as that in a case where the user inclines his head, that is, in Fig. 62, the position on the right end of the display screen, even after the user inclines his head. [0651]
As described above, in the terminal 12 (the display control unit 52 of the terminal 12), in a case where the AR display or the VR display is performed by using the agricultural field A as a background space, it is possible to display the entire object indicating the ecosystem constituent relevant to the entire agricultural field A in the fixed position on the display screen. [0652]
Accordingly, in a case where the AR display or the VR display is performed by using the agricultural field A as the background space, the user is capable of recognizing, for example, the concentration of a nitric acid, the moisture content, the climate, and the like, as the ecosystem constituent relevant to the entire agricultural field A, at an arbitrary timing. [0653]
Furthermore, in the entire object, for example, it is possible to turn the display On/Off according to the operation of the user. [0654]
[0655]
Fig. 63 is a diagram illustrating the display of the associated information in the terminal 12. [0656]
As described in Fig. 49, the ecosystem object can be

linked to associated information associated with the ecosystem constituent indicated by the ecosystem object. For example, information stored in the CMS database such as the Synecoculture (registered trademark) DB that the
Synecoculture (registered trademark) CMS 71 is capable of accessing, information which is the node of the relationship graph described in Fig. 24 to Fig. 41, and the like can be adopted as the associated information. [0657]
Then, for example, as described in Fig. 53, in the display of the time-lapse mode of Fig. 49, in a case where the user selects a certain ecosystem object as a selected object, in the terminal 12, it is possible to display the associated information linked to the selected object. [0658]
In Fig. 63, in the terminal 12, a certain ecosystem object is selected as a selected object from the display of the time-lapse mode (Fig. 49), according to the operation of the user. [0659]
In Fig. 63, the selected object is an ecosystem object indicating certain vegetation. [0660]
The terminal 12 transmits the selected object, for example, to the server 13 according to the operation or the like of the user, and the server 13 specifies vegetation as associated information associated with vegetation indicated by the selected object from the terminal 12 (hereinafter, also referred to as associated vegetation), from the relationship graph. [0661]

That is, in the server 13, the graph display control unit 72, for example, specifies vegetation (of a node) linked to the vegetation (of a node) indicated by the selected object, in a multi-partite graph such as a relationship graph (a bipartite graph), as the associated vegetation. [0662]
Further, in the server 13, the Synecoculture (registered trademark) CMS 71 searches information of the associated vegetation, that is, for example, the picture, the name, or the like of the associated vegetation from the CMS database, and transmits the information to the terminal 12 in the format of a webpage. [0663]
Accordingly, in the terminal 12, as described in Fig. 53, a list of the associated vegetation is displayed in the format of a webpage along with the information of the associated vegetation (the picture or the like). [0664]
Therefore, the user of the terminal 12 is capable of easily obtaining the information of the vegetation associated with the vegetation indicated by the selected object, by only selecting the selected object. [0665]
[0666]
Fig. 64 is a diagram illustrating an example of selecting the AR object to be subjected to the AR display or the VR display. [0667]
In the terminal 12, it is possible to suitably select, that is, change the AR object to be subjected to the AR display

or the VR display in the background space in a superimposed manner (hereinafter, also referred to as a display target object) on the display screen of the display unit 35. [0668]
For example, in the terminal 12, it is possible to select (change) the display target object according to the degree of proficiency of the user of the terminal 12 with respect to Synecoculture (registered trademark). [0669]
Here, the user inputs the profile of the user in advance to be registered in the terminal 12 or the server 13, and thus, the degree of proficiency of the user of the terminal 12 can be recognized on the basis of the profile. [0670]
A of Fig. 64 illustrates a display example of the AR display or the VR display for the user as a beginner. [0671]
In A of Fig. 64, ecosystem objects objES1, objES2, objES3, and objES4, and task objects objT1, objT2, objT3, objT12, and objT23 are displayed by using a certain agricultural field A as a background space. [0672]
The ecosystem objects objES1 and objES3 indicate the Chinese cabbage. The ecosystem object objES2 indicates the Chinese chive, and the ecosystem object objES4 indicates the wormwood. [0673]
The task objects objT1 to objT3 indicate the harvest of the vegetation, and the task objects objT12 and objT23 indicate the harvest order. [0674]

The user as a beginner observes the ecosystem objects objES1 to objES4, and thus, it is possible to easily recognize a position in the agricultural field A where vegetation such as the Chinese cabbage indicated by each of the ecosystem objects objES1 to objES4 exists. Further, the user as a beginner is capable of easily recognizing that the Chinese cabbage indicated by the ecosystem object objES1, the Chinese chive indicated by the ecosystem object objES2, and the Chinese cabbage indicated by the ecosystem object objES3 are planned to be harvested in this order, according to the task objects objT1 to objT3, and the task objects objT12 and objT23. [0675]
As described above, in the AR display or the VR display for the user as a beginner, in order to teach Synecoculture (registered trademark) to a beginner by so-called spoon-feeds, an ecosystem object indicating an ecosystem constituent of which the expertise is not so high, a task object suavely indicating a task performed with respect to the ecosystem constituent, or the like, is selected as the display target object, and is displayed. [0676]
B of Fig. 64 illustrates a display example of the AR display or the VR display for the user as a user as a skilled person. [0677]
In B of Fig. 64, the ecosystem object objES1, objES2, objES3, and objES4, and the task object objT1, objT2, and objT3 are displayed by using a certain agricultural field A as a background space. [0678]
Therefore, the AR display or the VR display in B of Fig.

64, are different from the AR display or the VR display for a beginner in A of Fig. 64 in which the task objects objT12 and objT23 are selected as the display target object, and are displayed, in that the task objects objT12 and objT23 are not displayed as the display target object. [0679]
The user as a skilled person, for example, obtains in advance that the harvest of the Chinese cabbage and the Chinese chive is planned to be performed in the order of the Chinese cabbage indicated by the ecosystem object objES1, the Chinese chive indicated by the ecosystem object objES2, and the Chinese cabbage indicated by the ecosystem object objES3, as knowledge. [0680]
For this reason, for the user as a skilled person, the task objects objT12 and objT23 indicating the harvest order of the Chinese cabbage and the Chinese chive are redundant information, and in the display of such task objects objT12 and objT23, there is a case where the user as a skilled person feels annoying. [0681]
Therefore, in the AR display or the VR display for a skilled person in B of Fig. 64, the ecosystem objects objES1 to objES4, and the task objects objT1 to objT3 are selected as the display target object, and are displayed, by excluding the task objects objT12 and objT23 from the ecosystem objects objES1 to objES4, and the task objects objT1 to objT3, objT12, and objT23. [0682]
Furthermore, for example, in Fig. 64, in a case where many types of vegetation exist in the agricultural field A as the background space, and an ecosystem object indicating

each of all types of vegetation existing in the agricultural field A, and all task objects indicating a task to be performed with respect to each of types of vegetation are displayed, there is a case where the number of ecosystem objects or task objects increases, and thus, it is difficult to observe the ecosystem objects or the task objects. [0683]
In addition, for example, rare vegetation is important for the user as a skilled person, but is not important for the user as a beginner. [0684]
Therefore, in the ecosystem object indicating the vegetation, in the terminal 12, for example, only an ecosystem object indicating the rare vegetation can be displayed to a user whose degree of proficiency is high, and only an ecosystem object indicating major vegetation can be displayed to a user whose degree of proficiency is low, according to the degree of proficiency of the user. [0685]
Furthermore, the display target object according to the degree of proficiency, for example, can be selected by allocating a level indicating the degree of proficiency into the AR object, and by selecting the AR object into which the level indicating the degree of proficiency of the user is allocated. [0686]
Fig. 65 is a diagram illustrating another example of selecting the AR object to be subjected to the AR display or the VR display. [0687]
As described in Fig. 64, the display target object can

be selected according to the degree of proficiency of the user, and can be selected by designating a category of data. [0688]
That is, a described in Fig. 44, the AR object, for example, can be divided into a picture (data of the picture), an icon (data of the icon), and sensor data (a symbol indicating the sensor data), according to the category of the data as the AR object. [0689]
In the terminal 12, for example, one or more categories can be selected from categories of the picture, the icon, and the sensor data as described above, and an AR object of the category can be selected as the display target object, according to the operation of the user. [0690]
A of Fig. 65 illustrates a display example of the AR display or the VR display in a case where AR objects of all of the categories of the picture, the icon, and the sensor data are selected as the display target object. [0691]
In A of Fig. 65, an ecosystem object of a picture indicating each of the Chinese cabbage and the Chinese chive, a task object of an icon indicating a task of harvesting the Chinese cabbage or the Chinese chive, and an ecosystem object of sensor data indicating each of the concentration of a nitric acid and a moisture content are selected as the display target object, and are displayed. [0692]
B of Fig. 65 illustrates a display example of the AR display or the VR display in a case where only the AR object of the category of the picture is selected from the categories

of the picture, the icon, and the sensor data, as the display
target object.
[0693]
In B of Fig. 65, the ecosystem object of the picture indicating each of the Chinese cabbage and the Chinese chive is selected as the display target object, and is displayed. [0694]
C of Fig. 65 illustrates a display example of the AR display or the VR display in a case where only the AR object of the category of the sensor data is selected from the categories of the picture, the icon, and the sensor data, as the display target object. [0695]
In C of Fig. 65, the ecosystem object of the sensor data indicating each of the concentration of the nitric acid and the moisture content is selected as the display target object, and is displayed. [0696]
D of Fig. 65 illustrates a display example of the AR display or the VR display in a case where the AR objects of the picture and the icon are selected from the categories of the picture, the icon, and the sensor data, as the display target object. [0697]
In D of Fig. 65, the ecosystem object of the picture indicating each of the Chinese cabbage and the Chinese chive, and the task object of the icon indicating the task of harvesting the Chinese cabbage or the Chinese chive are selected as the display target object, and are displayed. [0698]
As described above, in the terminal 12, for example,

it is possible to select one or more categories from the category of the AR object, and to select the AR object of the category as the display target object. Therefore, it is possible to display only the AR object of the category that the user requires, in the AR display or the VR display. [0699]
[0700]
Fig. 66 is a flowchart illustrating a processing example of registering the ecosystem object in the ecosystem utilization system of Fig. 1 (Fig. 43). [0701]
In step S311, the sensor device 11 or the terminal 12 acquires registration target information with respect to an ecosystem constituent to which an ecosystem object is applied (hereinafter, also referred to as an attention ecosystem constituent). [0702]
That is, the sensor device 11, for example, acquires sensor data obtained by sensing a predetermined physical amount of an atmospheric temperature, moisture, or the like, as the attention ecosystem constituent, as the registration target information. [0703]
In addition, in the acquisition unit 51, the terminal 12 (Fig. 3), for example, acquires a picture of an attention ecosystem constituent of vegetation or the like captured in an agricultural field, a comment input by the user (for example, "observe bees", "what is this vegetable organism?", or the like), as the registration target information, according to the operation of the user.

[0704]
In step S312, the sensor device 11 or the terminal 12 acquires positional information of the attention ecosystem constituent. [0705]
That is, the sensor device 11, for example, acquires GPS coordinates indicating the position of the sensor device 11 itself, as the positional information of the attention ecosystem constituent. [0706]
In addition, the terminal 12, for example, acquires the GPS coordinates indicating the position of the terminal 12 itself, as the positional information of the attention ecosystem constituent. [0707]
Furthermore, in the terminal 12, for example, is capable of detecting a relative position of the attention ecosystem constituent on the basis of the position of the terminal 12 itself, and of acquiring coordinates which is obtained by correcting the GPS coordinates of the terminal 12 itself with the relative position of the attention ecosystem constituent, as the positional information of the attention ecosystem constituent. [0708]
In addition, in the terminal 12, for example, the user operates the operation unit 34, and thus, it is possible to input the positional information of the attention ecosystem constituent. [0709]
In step S313, the sensor device 11 or the terminal 12 transmits the registration target information and the

positional information acquired with respect to the attention
ecosystem constituent, to the server 13.
[0710]
In step S321, the server 13 receives the registration target information and the positional information of the attention ecosystem constituent, which are transmitted from the sensor device 11 or the terminal 12. [0711]
In step S322, the generation unit 73 of the server 13 (Fig. 4) acquires the ecosystem object to which the attention ecosystem constituent is applied. [0712]
That is, in a case where a picture obtained by capturing the attention ecosystem constituent is included in the registration target information of the attention ecosystem constituent, the generation unit 73, for example, adopts the picture as the ecosystem object applied to the attention ecosystem constituent. [0713]
In addition, in a case where the sensor data is included in the registration target information of the attention ecosystem constituent, the generation unit 73 generates a symbol indicating the sensor data as the ecosystem object applied to the attention ecosystem constituent. [0714]
In addition, in the generation unit 73, for example, the ecosystem object of the icon indicating the bee-hive described in Fig. 52, the ecosystem object of the icon indicating the dangerous area, or the like, can be displayed on the terminal 12 as a candidate object of the ecosystem object applied to the attention ecosystem constituent.

[0715]
Then, in the generation unit 73, the candidate object, which is selected from the candidate object displayed on the terminal 12 by the user, can be adopted as the ecosystem object applied to the attention ecosystem constituent. [0716]
In step S323, the generation unit 73 generates object information associated with the registration target information of the attention ecosystem constituent, the positional information, the ecosystem object applied to the attention ecosystem constituent, and the like. Then, the generation unit 73 registers the object information, for example, in the object DB for registering the object information, stored in the storage 63. [0717]
Fig. 67 is a diagram illustrating the outline of the object information. [0718]
As described in Fig. 66, the object information includes the ecosystem object indicating the ecosystem constituent, the positional information of the ecosystem constituent, and the registration target information. [0719]
In addition, the object information is capable of including agricultural field information, marker information, a task object, or the like. [0720]
The agricultural field information is information associated with an agricultural in which an ecosystem constituent is observed (an agricultural field existing in a position indicated by the positional information). The

agricultural field information is capable of including an agricultural field name, an agricultural field address, information of the soil in the agricultural field, or the like. [0721]
According to the agricultural field information, for example, the object information is searched by using the agricultural field name as a keyword, and thus, it is possible to pick up an ecosystem object indicating an ecosystem constituent existing in a certain agricultural field. [0722]
The marker information is information associated with a marker displaying an ecosystem object. The marker information, for example, includes relative positional information indicating a relative position displaying the ecosystem object, or the like, on the basis of an image as the marker or the marker. [0723]
The image as the marker, for example, can be uploaded to the server 13 from the terminal 12. In addition, the image as the marker, for example, can be generated in the server 13, and can be downloaded to the terminal 12. [0724]
The relative positional information of the marker, for example, can be input according to the operation of the terminal 12 by the user, or can be calculated from the disposition position of the marker and the positional information of the ecosystem object. [0725]
In the terminal 12, in a case where the image as the marker included in the marker information is detected from an image captured with the camera 37, the ecosystem object

is displayed in a position indicated by the relative positional information included in the marker information, based on the marker. [0726]
Furthermore, in a plurality of object information items of which the positional information items are close to each other, the same image can be adopted as the image as the marker included in the marker information. In this case, one marker can be associated with a plurality of ecosystem objects, and the plurality of ecosystem objects indicating each of a plurality of ecosystem constituents in positions close to each other, can be displayed by one marker. [0727]
In the object information, the task object indicates a task performed with respect to the ecosystem constituent indicated by the ecosystem object included in the object information. [0728]
[0729]
Fig. 68 is a flowchart illustrating a processing example of displaying the AR object in the AR mode in the ecosystem utilization system of Fig. 1 (Fig. 43). [0730]
In step S331, the acquisition unit 51 of the terminal 12 (Fig. 3) acquires a real space which is a background space, a captured real space, or a VR space. [0731]
That is, for example, in a case where the terminal 12 is an immersive HMD or a smart phone, the captured real space obtained by capturing the real space with the camera 37, or

the VR space (an image of the VR space) in which the real space is modeled, is acquired as a background space (an image of the background space). [0732]
In addition, for example, in a case where the terminal 12 is a see-through HMD, a real space that the user of the terminal 12 observes in a see-through manner, is directly used as a background space. [0733]
In step S332, the acquisition unit 51 of the terminal
12 transmits ecosystem objects arranged in the background space
and an object request of requesting a task object to the server
13 (through the communication unit 40).
[0734]
That is, the acquisition unit 51, for example, detects a marker from the background space, and transmits the object request to the server 13 along with the marker. Alternatively, the acquisition unit 51 acquires the current location of the terminal 12, and transmits the object request to the server 13 along with the current location. [0735]
In step S341, the AR/VR display control unit 74 of the server 13 (Fig. 4) transmits the ecosystem object and the task object to the terminal 12 (through the communication unit 67), according to the object request which is transmitted from the terminal 12. [0736]
That is, the AR/VR display control unit 74 searches the object information including the marker which is transmitted from the terminal 12 along with the object request, or the object information including the positional information

indicating a position close to the current location of the terminal 12, which is transmitted from the terminal 12 along with the object request (hereinafter, also referred to as absolute positional information), from the object DB stored in the storage 63, as the attention object information. [0737]
Then, the AR/VR display control unit 74 transmits the ecosystem object included in the attention object information, the relative positional information of the marker, or the absolute positional information (hereinafter, also referred to as object positional information including both thereof) to the terminal 12. [0738]
Further, in a case where the attention object information includes the task object, the AR/VR display control unit 74 transmits the task object to the terminal 12. [0739]
In step S333, the acquisition unit 51 of the terminal 12 acquires the ecosystem object, the task object, or the object positional information, which is transmitted from the server 13, by receiving the ecosystem object, the task object, or the object positional information. [0740]
In step S334, the display control unit 52 of the terminal 12 performs the AR display with respect to the ecosystem object acquired in the acquisition unit 51 in a position of the background space acquired in step S331, which is indicated by the object positional information, in a superimposed manner. [0741]
Further, in a case where the acquisition unit 51 acquires the task object, the display control unit 52 performs the AR

display with respect to the task object acquired in the acquisition unit 51, in the vicinity of the ecosystem object in the background space in a superimposed manner. [0742]
As described above, in the terminal 12, for example, the AR display described in Fig. 44, Fig. 45, or the like is performed. [0743]
Furthermore, in the terminal 12, the object information including the absolute positional information indicating the position close to the current location of the terminal 12 can be constantly downloaded from the server 13. [0744]
In this case, in the terminal 12, the ecosystem object or the task object included in the object information which has been downloaded from the server 13, can be displayed without performing the object request with respect to the server 13, that is, without performing the communication with respect to the server 13. [0745]
Fig. 69 is a flowchart illustrating a processing example of displaying the AR object in the VR mode in the ecosystem utilization system of Fig. 1 (Fig. 43). [0746]
In step S351, the terminal 12 waits for the user to designate a predetermined position as a designated position, and transmits the designated position to the server 13. [0747]
In step S361, the server 13 receives the designated position from the terminal 12. [0748]

In step S362, the server 13 transmits a captured real space obtained in which a real space in a predetermined range including the designated position from the terminal 12 is captured, or a VR space (an image of the VR space) in which the real space is modeled, to the terminal 12. [0749]
That is, in the server 13, the storage 63 stores the captured real space in which the real space is captured, and VR space in which the real space is modeled. Then, in the server 13, the AR/VR display control unit 74 reads out the captured real space in which the real space including the designated position is captured, and the VR space in which the real space is modeled, from the captured real space or the VR space stored in the storage 63, and transmits the captured real space and the VR space to the terminal 12. [0750]
In step S352, the acquisition unit 51 of the terminal 12 acquires the captured real space or the VR space from the server 13, as a background space. [0751]
In step S353, the acquisition unit 51 of the terminal
12 transmits ecosystem objects arranged in the background space
and an object request of requesting a task object to the server
13.
[0752]
That is, the acquisition unit 51 detects a marker from the background space, and transmits the object request to the server 13 along with the marker. Alternatively, the acquisition unit 51 transmits the object request to the server
13 along with the designated position.
[0753]

In step S363, the AR/VR display control unit 74 of the server 13 transmits the ecosystem object and the task object, to the terminal 12, according to the object request which is transmitted from the terminal 12. [0754]
That is, the AR/VR display control unit 74 searches the object information including the marker which is transmitted from the terminal 12 along with the object request, or the object information including the absolute positional information indicating a position close to the designated position, which is transmitted from the terminal 12 along with the object request, from the object DB stored in the storage 63, as the attention object information. [0755]
Then, the AR/VR display control unit 74 transmits the ecosystem object included in the attention object information, the relative positional information of the marker as the object positional information, or the absolute positional information to the terminal 12. [0756]
Further, in a case where the attention object information includes the task object, the AR/VR display control unit 74 transmits the task object to the terminal 12. [0757]
Hereinafter, in steps S354 and S355, in the terminal 12, processing similar to that in each of steps S333 and S334 of Fig. 68 is performed, and thus, the ecosystem object or the task object is subjected to the VR display in the captured real space or the VR space as the background space acquired in step S352 in a superimposed manner. [0758]

As described above, in the terminal 12, for example, the VR display described in Fig. 46, Fig. 48, or the like is performed. [0759]
Furthermore, in Fig. 69, in the terminal 12, the ecosystem object or the task object is subjected to the VR display in the background space in a superimposed manner, but for example, in the server 13, an image can be generated in which the ecosystem object or the task object is superimposed in the background space, and is transmitted to the terminal 12, and in the terminal 12, the image transmitted from the server 13, can be displayed in which the ecosystem object or the task object are superimposed in the background space. [0760]
[0761]
Fig. 70 is a flowchart illustrating a processing example of editing the AR object in the ecosystem utilization system of Fig. 1 (Fig. 43). [0762]
That is, Fig. 70 illustrates a processing example in a case where, for example, a comment with respect to an ecosystem object is input, a task object indicating a task which is performed with respect to an ecosystem constituent indicated by the ecosystem object is associated with the ecosystem object, as the editing of the AR object. [0763]
In step S371, as described in Fig. 68 or Fig. 69, the terminal 12 displays the AR object in the AR mode or the VR mode. [0764]

In step S372, the acquisition unit 51 of the terminal 12 edits the AR object displayed in step S371, waits for the user to perform an operation of inputting editing information, and receives the editing information. [0765]
That is, in a case where the user, for example, performs the operation of inputting a comment with respect to the ecosystem object, as the editing of the AR object, the acquisition unit 51 receives the comment as the editing information. In addition, in a case where the user performs the operation of associating the task object indicating the task which is performed with respect to the ecosystem constituent indicated by the ecosystem object, with the ecosystem object, as the editing of the AR object, the acquisition unit 51 receives the task object as the editing information. [0766]
Here, in a case where the task object is associated with the ecosystem object, in the terminal 12, for example, it is possible to display a candidate of the task object which can be associated with the ecosystem object. The user is capable of selecting the task object associated with the ecosystem object from the candidate of the task object displayed on the terminal 12. In the terminal 12, the candidate of the task object, for example, can be acquired from the server 13. [0767]
In step S373, the acquisition unit 51 of the terminal 12 transmits the editing information received in step S372 to the server 13 along with identification information identifying the corresponding AR object. [0768]

Here, for example, the identification information for identifying the AR object is allocated into the AR object. In step S373, the terminal 12 transmits the editing information along with the identification information of the ecosystem object to which a comment is input, or the ecosystem object associated with the task object. [0769]
In step S381, the editing unit 75 of the server 13 receives the identification information and the editing information which are transmitted from the terminal 12. [0770]
In step S382, the editing unit 75 of the server 13 edits the AR object according to the editing information from the terminal 12. [0771]
That is, for example, in a case where the editing information is a comment, the editing unit 75 adds a comment as the editing information to the object information (Fig. 67) including the ecosystem object identified by the identification information which is transmitted from the terminal 12 along with the editing information. [0772]
In addition, for example, in a case where the editing information is a task object, the editing unit 75 adds a task object as the editing information to the object information including the ecosystem object identified by the identification information which is transmitted from the terminal 12 along with the editing information. [0773]
After that, in step S374, as described in Fig. 68 or Fig. 69, the terminal 12 displays again the AR object in the

AR mode or the VR mode. [0774]
That is, in the terminal 12, the acquisition unit 51 acquires again the AR object after being edited from the server 13, and the display control unit 52 displays the AR object after being edited. [0775]
As a result thereof, for example, in a case where the user performs the operation of inputting a comment with respect to the ecosystem object, in step S374, the ecosystem object is displayed along with the comment input by the user. [0776]
In addition, for example, in a case where the user performs the operation of associating the task object with the ecosystem object, in step S374, the ecosystem object is displayed along with the task object associated by the user. [0777]
Furthermore, the display of the comment with respect to the ecosystem object can be turned On/Off according to the operation of the user. [0778]
In addition, in Fig. 70, a case of editing the ecosystem object has been described, but the editing of the task object, that is, the inputting of a comment with respect to the task object can be performed in addition to the editing of the ecosystem object, as the editing of the AR object. [0779]
Fig. 71 is a flowchart illustrating another processing example of editing the AR object in the ecosystem utilization system of Fig. 1 (Fig. 43). [0780]

That is, Fig. 70 illustrates a processing example in a case where, for example, the ecosystem object is copied, as the editing of the AR object. [0781]
In step S391, as described in Fig. 69, the terminal 12, for example, displays the AR object in the VR mode, with respect to each of two agricultural fields A and B. [0782]
That is, the terminal 12 waits for the user to designate two agricultural fields A and B, and for example, as illustrated in Fig. 55, performs the VR display by using each of VR spaces of two agricultural fields A and B designated by the user as a background space. [0783]
In step S392, the acquisition unit 51 of the terminal 12 waits for the user to perform an operation of inputting copy information as the editing of the AR object displayed in step S391, and receives the copy information. [0784]
That is, as described in Fig. 55, the user, for example, is capable of copying an ecosystem object of a copy target to a desired position in a background space of the agricultural field B, by using a certain ecosystem object displayed in a background space of the agricultural field A as the copy target. [0785]
The acquisition unit 51 receives the copy object which is the ecosystem object of the copy target, and an agricultural field and a position of a copy destination to which the copy object is copied, as the copy information. [0786]
In step S393, the acquisition unit 51 of the terminal

12 transmits the copy information received in step S392 to
the server 13.
[0787]
In step S401, the editing unit 75 of the server 13 receives the copy information which is transmitted from the terminal 12. [0788]
In step S402, the editing unit 75 of the server 13 copies the ecosystem object according to the copy information from the terminal 12. [0789]
That is, the editing unit 75 generates new object information including the copy object which is included in the copy information, as the object information of the agricultural field of the copy destination included in the copy information. Further, the editing unit 75 sets positional information indicating the position of the copy destination included in the copy information, as (absolute) positional information of the new object information, and (adds) registers the new object information in the object DB of the storage 63. [0790]
In step S403, the editing unit 75 determines a task to be performed with respect to the ecosystem constituent indicated by the copy object. [0791]
For example, in a case where the ecosystem constituent indicated by the copy object is the bee-hive, the editing unit 75 determines that there is no task to be performed with respect to the ecosystem constituent indicated by the copy object. In this case, in the server 13, the subsequent processing is

not performed. [0792]
In addition, for example, in a case where the ecosystem constituent indicated by the copy object is vegetation, the editing unit 75 determines the planting of the vegetation as a task to be performed with respect to the ecosystem constituent indicated by the copy object. [0793]
In this case, the editing unit 75 includes (registers) a task object indicating the planting in the new object information which is added to and registered in the object DB in step S402, and thus, the task object is associated with the copy object included in the new object information. [0794]
After that, in step S394, as described in Fig. 69, the terminal 12 displays again the AR object in the VR mode. [0795]
That is, in the terminal 12, the acquisition unit 51 acquires again the AR object after being edited from the server 13, and the display control unit 52 displays the AR object after being edited. [0796]
As a result thereof, for example, in a case where the copy object is copied to a desired position in the background space of the agricultural field B by using a certain ecosystem object displayed in the background space of the agricultural field A as the copy object, in step S394, the copy object is displayed in a desired position in the background space of the agricultural field B. [0797]
Further, in the server 13, in a case where the task object

is associated with the copy object, in step S394, the task object associated with the copy object is also displayed along with the copy object. [0798]
[0799]
Fig. 72 is a diagram illustrating a transition example of the display screen of the display unit 35 in a case where the AR object is registered or edited in the terminal 12. [0800]
The terminal 12 exchanges necessary information with respect to the server 13, and thus, the display screen of the display unit 35 is transitioned as illustrated in Fig. 72. [0801]
In the terminal 12, in a case where the user performs a predetermined operation, a screen of "Main Menu" is displayed on the display screen of the display unit 35. [0802]
The screen of "Main Menu" includes a button of [Camera Capture] and a button of [Edit Object]. [0803]
In a case where the button of [Camera Capture] of the screen of "Main Menu" is operated, the screen of "Capture" including a button of [Capture] is displayed on the display screen of the display unit 35. [0804]
In a case where the button of [Capture] of the screen of "Capture" is operated, in the terminal 12, a picture is captured by the camera 37, and the GPS coordinates (the positional information) are acquired at the time of capturing the picture. Then, a screen of "Comment Input" of inputting

a comment with respect to the picture is displayed on the display
screen of the display unit 35.
[0805]
The screen of "Comment Input" includes a button of [Registration], and in a case where the user inputs a comment, and operates the button of [Registration], a screen of "Registration Completion" including a button of [Completion] is displayed on the display screen of the display unit 35. [0806]
In a case where the button of [Completion] of the screen of "Registration Completion" is operated, the user transmits the picture captured on screen of "Capture" and the comment input on the screen of "Comment Input" to the server 13 from the terminal 12, as the registration target information described in Fig. 66. Further, the GPS coordinates acquired at the time of capturing the picture is transmitted to the server 13 from the terminal 12, as the positional information described in Fig. 66. Then, in the server 13, the object information including the positional information and the registration target information from the terminal 12 is registered. [0807]
After that, the display screen of the display unit 35 is returned to the screen of "Capture". [0808]
In a case where the button of [Edit Object] of the screen of "Main Menu" is operated, a screen of "Action List" is displayed on the display screen of the display unit 35. [0809]
The screen of "Action List" includes a button of [Generation], a button of [Browse], a button of [Task

Generation], a button of [Task List], a button of [Data Import],
and a button of [Registration Data].
[0810]
In a case where the button of [Generation] is operated on the screen of "Action List", a screen of "Coordinates Designation" including a button of [Designation] is displayed on the display screen of the display unit 35. [0811]
In a case where the user inputs coordinates and operates [Designation Button] on the screen of "Coordinates Designation", the screen of "Comment Input" including the button of [Registration] is displayed on the display screen of the display unit 35. [0812]
In a case where the user inputs a comment and operates the button of [Registration] on the screen of "Comment Input", the screen of "Registration Completion" including the button of [Completion] is displayed on the display screen of the display unit 35. [0813]
In a case where the button of [Completion] of the screen of "Registration Completion" is operated, the user transmits the coordinates input on the screen of "Coordinates Designation" to the server 13 from the terminal 12, as the positional information described in Fig. 66. Further, the comment input on the screen of "Comment Input" by the user is transmitted to the server 13 from the terminal 12, as the registration target information described in Fig. 66. Then, in the server 13, the object information including the positional information and the registration target information from the terminal 12 is registered. That is, in

this case, object information not including a picture as an ecosystem object, in other words, so-called object information including an empty picture is registered. [0814]
After that, the display screen of the display unit 35 is returned to the screen of "Coordinates Designation". [0815]
In a case where the button of [Browse] is operated on the screen of "Action List", a screen of "Object Selection" including a button of [Selection] is displayed on the display screen of the display unit 35. [0816]
For example, a list of pictures registered in the object DB of the server 13 as the ecosystem object is displayed on the screen of "Object Selection". [0817]
In a case where the user selects a picture as a certain ecosystem object and operates the button of [Selection] on the screen of "Object Selection", a screen of "Time Lapse-Relation Record" is displayed on the display screen of the display unit 35. [0818]
A time-lapse mode of the ecosystem object, in which an agricultural field where an ecosystem constituent indicated by the ecosystem object selected on the screen of "Object Selection" by the user is observed (the agricultural field indicated by the agricultural field information included in the object information of the ecosystem object selected by the user (Fig. 67)), is displayed on the screen of "Time Lapse-Relation Record", as the attention agricultural field described in Fig. 49.

[0819]
The screen of "Time Lapse-Relation Record" includes a button of [Object Selection], a button of [Other Agricultural Farm Name Selection], a button of [Capture], a button of [Comment Editing], and a button of [Deletion]. [0820]
In a case where on the user selects a certain ecosystem object as a selected object and operates the button of [Object Selection] on the screen of "Time Lapse-Relation Record", the server 13 generates the relationship graph using the node of the ecosystem constituent indicated by the selected object as the node of interest (Fig. 26 or the like), and a screen of "Gephigraph Model Search" including the relationship graph is displayed on the display screen of the display unit 35 of the terminal 12 by using Gephi. [0821]
The screen of "Gephigraph Model Search" includes a button of [URL Selection], and in a case where the user selects the node of the relationship graph and operates the button of [URL Selection] on the screen of "Gephigraph Model Search", a screen of "CMS Database Search" is displayed on the display screen of the display unit 35. [0822]
For example, as described in Fig. 53, associated information associated with vegetation or the like indicated by the node selected on the screen of "Gephigraph Model Search" by the user is searched from the CMS database of the server 13 and is displayed on the screen of "CMS Database Search". [0823]
In a case where the user selects a certain ecosystem object as a selected object and operates a button of [Other

Farm Name Selection] on the screen of "Time Lapse-Relation Record", for example, the same vegetation agricultural field information display illustrated in Fig. 50 is displayed on the display screen of the display unit 35. [0824]
Then, in a case where the user selects an agricultural field other than the attention agricultural field in which the ecosystem constituent indicated by the selected object exists, in the same vegetation agricultural field information display of Fig. 50, as a selected agricultural field, the display screen of the display unit 35, for example, is switched to a screen of "Target Agricultural Farm Target Object" through transition display using Google Earth illustrated in Fig. 51. [0825]
For example, the selected agricultural field as illustrated in Fig. 52 is displayed on the screen of "Target Agricultural Farm Target Object". [0826]
In a case where the user selects a certain ecosystem object as a selected object and operates the button of [Capture] on the screen of "Time Lapse-Relation Record", the screen of "Comment Input" is displayed on the display screen of the display unit 35. [0827]
The user is capable of performing an operation of replacing a picture as a selected object (including the empty picture) with the other picture on the screen of "Comment Input". [0828]
The screen of "Comment Input" includes a button of [Registration]. In a case where the user replaces the picture

as the selected object with the other picture and operates the button of [Registration] on the screen of "Comment Input", in the server 13, the picture as the selected object included in the object information (Fig. 67) is replaced. [0829]
Then, the screen of "Registration Completion" indicating that the replacement of the picture as the selected object is completed is displayed on the display screen of the display unit 35. [0830]
In a case where the user selects a certain ecosystem object as a selected object and operates a button of [Comment Editing] on the screen of "Time Lapse-Relation Record", a screen of "Editing" is displayed on the display screen of the display unit 35. [0831]
The user is capable of performing an operation of editing the comment included in the object information (Fig. 67) of the selected object on the screen of "Editing". [0832]
The screen of "Editing" includes a button of [Registration]. In a case where the user edits the comment included in the object information of the selected object and operates the button of [Registration] on the screen of "Editing", in the server 13, an editing result of the comment is reflected on the object information (Fig. 67). [0833]
Then, the screen of "Registration Completion" indicating that the editing of the comment is completed is displayed on the display screen of the display unit 35. [0834]

In a case where the user selects a certain ecosystem object as a selected object and operates the button of [Deletion] on the screen of "Time Lapse-Relation Record", the screen of "Deletion Confirmation" is displayed on the display screen of the display unit 35. [0835]
The screen of "Deletion Confirmation" includes a button of [Deletion]. In a case where the user operates the button of [Deletion] on the screen of "Deletion Confirmation", in the server 13, the object information of the selected object (Fig. 67) is deleted. [0836]
Then, the screen of "Deletion Completion" indicating that the deletion of the object information is completed is displayed on the display screen of the display unit 35. [0837]
In a case where the button of [Task Generation] is operated on the screen of "Action List", a screen of "Icon List" is displayed on the display screen of the display unit 35. [0838]
The screen of "Icon List" includes a button corresponding to a task which can be performed with respect to an ecosystem constituent. [0839]
In Fig. 72, in order to simplify the description, two tasks of harvest and tree trimming are adopted as the task which can be performed with respect to the ecosystem constituent, and thus, the screen of "Icon List" includes a button of [Harvest] indicating harvest and a button of [Weed Control] indicating tree trimming.

[0840]
In a case where the user operates the button of [Harvest] on the screen of "Icon List", a screen of "Object Designation" including a button of "Designation" is displayed on the display screen of the display unit 35. [0841]
An ecosystem object indicating an ecosystem constituent capable of being a target of harvest which is a task corresponding to the button of [Harvest] is displayed on the screen of "Object Designation". In a case where the user selects an ecosystem object and operates the button of "Designation" on such a screen of "Object Designation", a screen of "Comment Editing" including a button of [Registration] is displayed on the display screen of the display unit 35. [0842]
The user is capable of inputting a comment with respect to a task object indicating a task corresponding to the button of [Harvest], associated with the ecosystem object selected on the screen of "Object Designation", (hereinafter, also referred to as a harvest object) on the screen of "Comment Editing". [0843]
In a case where the user inputs the comment with respect to the harvest object and operates the button of [Registration] on the screen of "Comment Editing", in the server 13, the harvest object and the comment input by the user are (added to) registered in the object information of the ecosystem object selected on the screen of "Object Designation" by the user (Fig. 67). [0844]

Then, the screen of "Registration Completion" indicating that the registration of the harvest object is completed is displayed on the display screen of the display unit 35. [0845]
The screen of "Registration Completion" includes a button of [Completion], and in a case where the user operates the button of [Completion], the display screen of the display unit 35 is returned to the screen of "Icon List". [0846]
In a case where the user operates the button of [Weed Control] on the screen of "Icon List", the screen of "Object Designation" including a button of "Designation" is displayed on the display screen of the display unit 35. [0847]
An ecosystem object indicating an ecosystem constituent capable of being a target of tree trimming which is a task corresponding to the button of [Weed Control] is displayed on the screen of "Object Designation". In a case where the user selects the ecosystem object and operates the button of "Designation" on such a screen of "Object Designation", the screen of "Comment Editing" including the button of [Registration] is displayed on the display screen of the display unit 35. [0848]
The user is capable of inputting a comment with respect to a task object indicating the task corresponding to the button of [Weed Control], associated with the ecosystem object selected on the screen of "Object Designation", (hereinafter, also referred to as a tree trimming object), on the screen of "Comment Editing".

[0849]
In a case where the user inputs the comment with respect to the tree trimming object and operates the button of [Registration] on the screen of "Comment Editing", in the server 13, the tree trimming object and the comment input by the user are (added to) registered in the object information of the ecosystem object selected on the screen of "Object Designation" by the user (Fig. 67). [0850]
Then, the screen of "Registration Completion" indicating that the registration of the tree trimming object is completed, is displayed on the display screen of the display unit 35. [0851]
The screen of "Registration Completion" includes a button of [Completion], and in a case where the user operates the button of [Completion], the display screen of the display unit 35 is returned to the screen of "Icon List". [0852]
In a case where the button of [Task List] is operated on the screen of "Action List", a screen of "Registration Task List" is displayed on the display screen of the display unit 35. [0853]
The button of [Task Generation] of the screen of "Action List" is operated on the screen of "Registration Task List", and thus, a list of the registered task objects is displayed as a button to be operable. [0854]
Here, as described in the screen of "Icon List", in this embodiment, there are two types of harvest and tree trimming

as a task which can be performed with respect to an ecosystem constituent, and a task object indicating the harvest or the tree trimming is registered by being associated with the ecosystem object selected on the screen of "Object Designation" by the user. [0855]
As described above, the task object indicating the harvest or the tree trimming, registered by being associated with the ecosystem object, is displayed on the screen of "Registration Task List" in the format of a button. [0856]
For convenience, the button of the task object indicating the harvest or the tree trimming will be referred to as a button of [Harvest] and a button of [Weed Control], respectively. [0857]
In a case where a certain button of [Harvest] is operated on the screen of "Registration Task List", a screen of "Harvest Instruction" is displayed on the display screen of the display unit 35. [0858]
For example, an icon as the task object indicating the harvest, an arrow-like icon indicating a harvest order, or the like, as described in Fig. 47, is displayed with respect to the harvest indicated by the button of [Harvest] operated on the screen of "Registration Task List", and thus, it is visually presented that the harvest is started from which location, and is performed in which order, on the screen of "Harvest Instruction". [0859]
Furthermore, in the terminal 12, the user is capable of collectively designating the harvest order of the ecosystem

constituent at the time of selecting the ecosystem object indicating the ecosystem constituent which is the target of the harvest, on the screen of "Object Designation" displayed when the button of [Harvest] of "Icon List" is operated. Then, in the server 13, the arrow-like icon indicating the harvest order can be generated in the harvest order of the ecosystem constituent designated by the user. [0860]
The icon as the task object indicating the harvest is displayed on the screen of "Harvest Instruction", as a button of [Harvest Icon] to be operable. [0861]
In a case where a certain button of [Harvest Icon] is operated on the screen of "Harvest Instruction", a screen of "Details" is displayed on the display screen of the display unit 36. [0862]
Detailed information of the harvest indicated by the task object corresponding to the button of [Harvest Icon] operated on the screen of "Harvest Instruction" is displayed on the screen of "Details". [0863]
The detailed information of the harvest displayed on the screen of "Details", for example, includes a species name of a fruit (vegetation) to be harvested, a standard for selecting fruits to be harvested, such as harvesting red ripe fruits and allowing blue underripe fruits to remain, balance information with the other operation such as allowing 20% of the fruits for in-house seed to remain without harvesting all of the fruits, and the like. [0864]

Such detailed information, for example, can be input as a comment, on the screen of "Comment Editing" of inputting the comment with respect to the harvest object. [0865]
In a case where a certain button of [Weed Control] is operated on the screen of "Registration Task List", a screen of "Tree Trimming Instruction" is displayed on the display screen of the display unit 35. [0866]
For example, the icon as the task object indicating the tree trimming, the arrow-like icon indicating a tree trimming order, or the like, is displayed with respect to the tree trimming indicated by the button of [Weed Control] operated on the screen of "Registration Task List", and thus, it is visually presented that the tree trimming is started from grass or trees in which location, and is performed in which order, on the screen of "Tree Trimming Instruction". [0867]
Furthermore, in the terminal 12, the user is capable of collectively designating the tree trimming order of the ecosystem constituent at the time of selecting the ecosystem object indicating the ecosystem constituent which is the target of the tree trimming, on the screen of "Object Designation" displayed when the button of [Weed Control] of "Icon List" is operated. Then, in the server 13, the arrow-like icon indicating the tree trimming order can be generated in the tree trimming order of the ecosystem constituent designated by the user. [0868]
The icon as the task object indicating the tree trimming is displayed on the screen of "Tree Trimming Instruction",

as a button of [Tree Trimming Icon] to be operable. [0869]
In a case where a certain button of [Tree Trimming Icon] is operated on the screen of "Tree Trimming Instruction", the screen of "Details" is displayed on the display screen of the display unit 36. [0870]
Detailed information of the tree trimming indicated by the task object corresponding to the button of [Tree Trimming Icon] operated on the screen of "Tree Trimming Instruction" is displayed on the screen of "Details". [0871]
The detailed information of the tree trimming displayed on the screen of "Details", for example, includes a species name of a fruit (vegetation) to be subjected to tree trimming, a tree trimming method such as root elimination, mowing at the level of the land surface, mowing at the level of the height of the vegetable, and performing which vegetation strategy after the mowing, balance information with the other operation, and the like. [0872]
Such detailed information, for example, can be input as a comment, on the screen of "Comment Editing" of inputting the comment with respect to the tree trimming object. [0873]
In a case where the button of [Data Import] is operated on the screen of "Action List", a screen of "Import Data List" is displayed on the display screen of the display unit 35. [0874]
A list of sensor data items managed by a CMS other than the Synecoculture (registered trademark) CMS 71, and a button

of [Selection] are displayed on the screen of "Import Data
List".
[0875]
In a case where the user selects certain sensor data and operates the button of [Selection] on the screen of "Import Data List", a screen of [Import] is displayed on the display screen of the display unit 35. [0876]
Then, sensor data selected on the screen of "Import Data List" by the user is imported to the object DB of the storage 63 of the server 13. [0877]
A screen of "Import" includes a button of [Next], and in a case where the button of [Next] is operated, a screen of [Registration Data List] is displayed on the display screen of the display unit 35. [0878]
Here, the screen of [Registration Data List] is also displayed in a case where the button of [Registration Data] is operated on the screen of "Action List". [0879]
For example, each agricultural field (an agricultural field name of each of the agricultural fields), a button of [Nitric Acid], a button of [Electric Conductivity], and a button of [Soil Hardness] are displayed on the screen of [Registration Data List]. [0880]
In a case where the user selects a certain agricultural field and operates the button of [Nitric Acid] on the screen of [Registration Data List], a screen of "Visual Data Display" of a concentration of a nitric acid is displayed on the display

screen of the display unit 35. [0881]
Sensor data of the concentration of a nitric acid (a symbol of the sensor data) sensed in the agricultural field which is selected on the screen of [Registration Data List] by the user, is displayed on the screen of "Visual Data Display" of the concentration of a nitric acid. [0882]
In a case where user selects a certain agricultural field and operates the button of [Electric Conductivity] on the screen of [Registration Data List], a screen of "Visual Data Display" of an electric conductivity is displayed on the display screen of the display unit 35. [0883]
Sensor data of the electric conductivity sensed in the agricultural field which is selected on the screen of [Registration Data List] by the user is displayed on the screen of "Visual Data Display" of the electric conductivity. [0884]
In a case where the user selects a certain agricultural field and operates the button of [Soil Hardness] on the screen of [Registration Data List], a screen of "Visual Data Display" of a soil hardness is displayed on the display screen of the display unit 35. [0885]
Sensor data of the soil hardness sensed in the agricultural field which is selected on the screen of [Registration Data List] by the user is displayed on the screen of "Visual Data Display" of the soil hardness. [0886]
Here, the processing performed by a computer (CPU)

according to a program is not necessarily performed in chronological order along the order described herein as a flowchart. That is, the processing performed by the computer according to the program also includes processing executed in parallel or individually (for example, parallel processing or processing according to an object). [0887]
In addition, the program may be processed by one computer (processor), or may be subjected to distribute processing by a plurality of computers. Further, the program may be executed by being transmitted to a distant computer. [0888]
Further, herein, the system indicates an aggregation of a plurality of constituents (a device, a module (a component), or the like), and it does not matter whether or not all of the constituents are in the same housing. Therefore, both of a plurality of devices which are contained in an individual housing and are connected to each other through a network, and one device in which a plurality of modules are contained in one housing are the system. [0889]
Furthermore, the embodiment of the present technology is not limited to the embodiment described above, but various modifications can be performed within a range not departing from the gist of the present technology. That is, the present technology can be applied to general ecosystem management in addition to the assistance of Synecoculture (registered trademark). [0890]
Further, for example, the present technology is capable of including a cloud computing configuration of performing

processing by sharing and collaborating one function in a
plurality of devices through a network.
[0891]
In addition, each step described in the flowchart can be executed in one device, and can be executed by being shared in a plurality of devices. [0892]
Further, in a case where a plurality of processings are included in one step, the plurality of processings included in one step can be executed in one device, and can be executed by being shared in a plurality of devices. [0893]
In addition, the effect described herein is merely an example, but is not limited thereto, and other effects may be obtained. [0894]
Furthermore, the present technology is capable of including the following configurations. [0895]
<1>
An information processing device, including:
an acquisition unit that acquires an ecosystem object indicating an ecosystem constituent configuring an ecosystem of an agricultural field in which a plurality of types of vegetation are mixed up, and a task object indicating a task performed with respect to the ecosystem constituent; and
a display control unit that performs display control of performing augmented reality (AR) display of the ecosystem object in a position in a predetermined background space, corresponding to a real position of the ecosystem constituent, and performing AR display of the task object in the background

space.
<2>
The information processing device according to <1>,
in which a predetermined marker or a positional information of a global positioning system (GPS) is associated with the ecosystem object, and
the display control unit performs display control of displaying the ecosystem object in a relative position based on the marker or a position indicated by the positional information in the background space.
<3>
The information processing device according to <1> or <2>,
in which the background space is a real space of a reality, a captured real space in which the real space is captured, or a virtual reality (VR) space in which the real space is modeled.
<4>
The information processing device according to any one of <1> to <3>,
in which the ecosystem object or the task object is edited according to an operation of a user, and
the acquisition unit acquires the ecosystem object and the task object after being edited.
<5>
The information processing device according to <4>,
in which the ecosystem object of one agricultural field is copied on another agricultural field as editing of the ecosystem object.
<6>
The information processing device according to <5>,

in which a task object indicating a task to be performed with respect to an ecosystem constituent indicated by the ecosystem object to be copied on the another agricultural field is added.
<7>
The information processing device according to any one of <1> to <6>,
in which a display scale of the background space is capable of being changed.
<8>
The information processing device according to any one of <1> to <7>,
in which images of two or more agricultural fields are capable of being simultaneously displayed as the background space.
<9>
The information processing device according to <2>,
in which one marker is associated with the ecosystem object of a part or all of one agricultural field.
<10>
The information processing device according to <2>,
in which two different markers are associated with the same ecosystem object.
<11>
The information processing device according to <2>,
in which the marker associated with the ecosystem object is movable, and
the ecosystem object is displayed in the same position as the position before the marker is moved, after the marker associated with the ecosystem object is moved.
<12>

The information processing device according to any one of <1> to <11>,
in which the display control unit further performs display control of displaying the ecosystem object along one axis.
<13>
The information processing device according to <12>,
in which the display control unit further performs display control of displaying the ecosystem object along a time axis.
<14>
The information processing device according to any one of <1> to <13>,
in which the ecosystem object is a picture in which the ecosystem constituent is captured, a symbol indicating sensor data which is obtained by sensing the ecosystem constituent, or an icon indicating the ecosystem constituent.
<15>
The information processing device according to any one of <1> to <14>,
in which the display control unit further performs display control of displaying the ecosystem object indicating the ecosystem constituent relevant to the entire agricultural field in a fixed position.
<16>
The information processing device according to any one of <1> to <15>,
in which associated information associated with the ecosystem constituent which is indicated by the ecosystem object is linked to the ecosystem object, and,
the display control unit further performs display

control of displaying the associated information.
<17>
The information processing device according to any one of <1> to <16>,
in which the ecosystem object or the task object to be displayed, is changed according to a degree of proficiency of a user.
<18>
The information processing device according to any one of <1> to <17>,
in which the ecosystem constituent includes a dangerous area or an area to be watched of the agricultural field.
<19>
The information processing device according to any one of <1> to <18>,
in which ecosystem objects of one or more categories selected from a plurality of categories are displayed.
<20>
An information processing method, including:
acquiring an ecosystem object indicating an ecosystem constituent configuring an ecosystem of an agricultural field in which a plurality of types of vegetation are mixed up, and a task object indicating a task performed with respect to the ecosystem constituent; and
performing display control of performing augmented reality (AR) display of the ecosystem object in a position in a predetermined background space, corresponding to a real position of the ecosystem constituent, and performing AR display of the task object in the background space.
<21>
A program for allowing a computer to function as:

an acquisition unit acquiring an ecosystem object indicating an ecosystem constituent configuring an ecosystem of an agricultural field in which a plurality of types of vegetation are mixed up, and a task object indicating a task performed with respect to the ecosystem constituent; and
a display control unit performing augmented reality (AR) display of the ecosystem object in a position in a predetermined background space, corresponding to a real position of the ecosystem constituent, and display control of AR display of the task object in the background space.
REFERENCE SIGNS LIST [0896]
10 Network
11 Sensor device
12 Terminal
13 Server

21 Wireless relay device
22 Wireless LAN
23 Mobile phone network
24 Internet

31 CPU
32 Memory
33 Storage
34 Operation unit
35 Operation unit
36 Speaker
37 Camera
38 Microphone
39 Position detection unit
40 Communication unit

41 External I/F
42 Drive
42A Removable recording medium
43 Sensor
51 Acquisition unit
52 Display control unit

61 CPU
62 Memory
63 Storage
64 Operation unit
65 Display unit
66 Speaker
67 Communication unit
68 External I/F
69 Drive
69A Removable recording medium
71 Synecoculture (registered trademark) CMS
72 Graph display control unit
73 Generation unit
74 AR/VR display control unit
75 Editing unit
76 Analysis unit
101 to 104 Chinese cabbage
105 Chinese chive
106 Radish
107 Cauliflower
108 Japanese mustard spinach
109 Burdock root
110 Wormwood stand zone
121 Flag
122 to 133 Icon

201 Graph display region
202 List display region

211 Overview display region
212 Display frame

CLAIMS
1. An information processing device, comprising:
an acquisition unit that acquires an ecosystem object indicating an ecosystem constituent configuring an ecosystem of an agricultural field in which a plurality of types of vegetation are mixed up, and a task object indicating a task performed with respect to the ecosystem constituent; and
a display control unit that performs display control of performing augmented reality (AR) display of the ecosystem object in a position in a predetermined background space, corresponding to a real position of the ecosystem constituent, and performing AR display of the task object in the background space.
2. The information processing device according to claim
1,
wherein a predetermined marker or a positional information of a global positioning system (GPS) is associated with the ecosystem object, and
the display control unit performs display control of displaying the ecosystem object in a relative position based on the marker or a position indicated by the positional information in the background space.
3. The information processing device according to claim
1,
wherein the background space is a real space of a reality, a captured real space in which the real space is captured, or a virtual reality (VR) space in which the real space is modeled.

4. The information processing device according to claim
1,
wherein the ecosystem object or the task object is edited according to an operation of a user, and
the acquisition unit acquires the ecosystem object and the task object after being edited.
5. The information processing device according to claim
4,
wherein the ecosystem object of one agricultural field is copied on another agricultural field as editing of the ecosystem object.
6. The information processing device according to claim
5,
wherein a task object indicating a task to be performed with respect to an ecosystem constituent indicated by the ecosystem object to be copied on the another agricultural field is added.
7. The information processing device according to claim
1,
wherein a display scale of the background space is capable of being changed.
8. The information processing device according to claim
1,
wherein images of two or more agricultural fields are capable of being simultaneously displayed as the background space.

9. The information processing device according to claim
2,
wherein one marker is associated with the ecosystem object of a part or all of one agricultural field.
10. The information processing device according to claim
2,
wherein two different markers are associated with the same ecosystem object.
11. The information processing device according to claim
2,
wherein the marker associated with the ecosystem object is movable, and
the ecosystem object is displayed in the same position as the position before the marker is moved, after the marker associated with the ecosystem object is moved.
12. The information processing device according to claim
1,
wherein the display control unit further performs display control of displaying the ecosystem object along one axis.
13. The information processing device according to claim
12,
wherein the display control unit further performs display control of displaying the ecosystem object along a time axis.

14. The information processing device according to claim
1,
wherein the ecosystem object is a picture in which the ecosystem constituent is captured, a symbol indicating sensor data which is obtained by sensing the ecosystem constituent, or an icon indicating the ecosystem constituent.
15. The information processing device according to claim
1,
wherein the display control unit further performs display control of displaying the ecosystem object indicating the ecosystem constituent relevant to the entire agricultural field in a fixed position.
16. The information processing device according to claim
1,
wherein associated information associated with the ecosystem constituent which is indicated by the ecosystem object is linked to the ecosystem object, and
the display control unit further performs display control of displaying the associated information.
17. The information processing device according to claim
1,
wherein the ecosystem object or the task object to be displayed, is changed according to a degree of proficiency of a user.
18. The information processing device according to claim
1,
wherein the ecosystem constituent includes a dangerous

area or an area to be watched of the agricultural field.
19. The information processing device according to claim
1,
wherein ecosystem objects of one or more categories selected from a plurality of categories are displayed.
20. An information processing method, comprising:
acquiring an ecosystem object indicating an ecosystem
constituent configuring an ecosystem of an agricultural field in which a plurality of types of vegetation are mixed up, and a task object indicating a task performed with respect to the ecosystem constituent; and
performing display control of performing augmented reality (AR) display of the ecosystem object in a position in a predetermined background space, corresponding to a real position of the ecosystem constituent, and performing AR display of the task object in the background space.