High Level Objects

This tutorial discusses some concepts that are important when working with high level objects in resqpy.

RESQML and resqpy classes

The RESQML standard defines many classes of high level objects and specifies precisely how they are to be represented in persistent storage (files). However, application code making use of resqpy will not usually interact directly with the RESQML objects but rather with the closely related resqpy classes of object. Whilst there is a degree of correspondence between RESQML high level classes and resqpy classes, there are some differences which should be borne in mind:

  • Class names are usually different

  • Some resqpy classes cater for more than one RESQML class

  • There are a few circumstances where a RESQML object can be represented by more than one resqpy class

  • RESQML is purely concerned with what data is stored for a class, whilst a resqpy class also contains methods to provide different ways of viewing or processing the data

  • Whereas a RESQML class is defined in a hierarchical way, and makes use of inheritance (xsd extension base) and abstract classes, the comparable resqpy class is flattened with data elements held as simple attributes

  • Some resqpy classes use class inheritance to allow common functionality to be implemented in a base class – this is a different hierarchy to that used in the RESQML schema definition

  • Not all RESQML classes are yet catered for (except in the lowest level generic layer of code)

  • Some RESQML objects have optional attributes or multiple possible representations of an attribute – some of the options might not yet be implemented in resqpy

Apart from the last two of these points, the differences are due to the different aims of RESQML and resqpy: RESQML aims to provide a comprehensive and unambiguous standard for efficient storage of reservoir models, whereas resqpy aims to provide high level functionality to facilitate processing of the models.

The table at the end of this page shows which resqpy class implements each RESQML class.

Reading and writing objects

From the discussion above, it is evident that the same information can exist in two different representations: in a file in RESQML format, or in memory as resqpy objects. When reading a dataset, the transformation is from RESQML to resqpy. When writing, the transformation is from resqpy to RESQML. However, for efficiency of processing, things are more complicated than that and the representation of a conceptual object can exist in one of a number of states.

Firstly, the resqpy code differentiates between RESQML classes depending on how much array data they involve:

  • Classes with no array data, for example measured depth datum (obj_MdDatum in RESQML, MdDatum in resqpy)

  • Classes with modest amounts of array data, eg. wellbore trajectory (obj_WellboreTrajectoryRepresentation in RESQML, Trajectory in resqpy)

  • Classes with large amounts of array data, eg. ijk cellullar grid (obj_IjkGridRepresentation in RESQML, Grid in resqpy)

The rest of this tutorial will refer to these volumes of array data as none, small or large respectively. Note that the behaviour of the resqpy code is based on the typical amounts of array data for a given class, not the actual size of the arrays for a specific object.

When reading, the representation of an object passes through these states:

  1. Only in files: metadata in xml compressed into the epc file; any array data in the hdf5 file

  2. Metadata loaded into equivalent data structure in memory; any array data still only in the hdf5 file

  3. In memory resqpy object instantiated; metadata in object attributes; if small array(s), array data also in memory as attributes

  4. For classes with large arrays, Individual arrays are cached as attributes on demand

Step 2 in this sequence occurs with the instantiation of a Model object for an existing epc. The metadata for each part is loaded into a Python lxml tree (which is compatible with elementTree). Application code does not usually interact directly with this representation, though the root node of the lxml tree for an object is sometimes used as an argument to resqpy function calls. Here is an example of code that moves all objects in the s_bend dataset into state 2:

import resqpy.model as rq
model = rq.Model('s_bend.epc')

Step 3 occurs when the application code instantiates a resqpy object for one of the parts in the model. At this point, the lxml metadata is interrogated to set the values of the class-specific attributes. The naming and definition of these attributes is often very similar to the equivalent metadata fields in the RESQML class. If the class has a small volume of array data, then it is also loaded at this point into numpy array attributes. The resqpy class might also have derived attributes which are not stored in the RESQML object but are set for the convenience of application code. The following lines will create a resqpy Grid object in state 3 for one of the IjkGridRepresentation parts in the s_bend model:

import resqpy.grid as grr
faulted_grid_uuid = model.uuid(obj_type = 'IjkGridRepresentation', title = 'FAULTED GRID')
faulted_grid = grr.Grid(model, uuid = faulted_grid_uuid)

Step 4 only pertains to classes with large amounts of array data. To minimize memory and time usage, these arrays are not loaded until application code requests them using specific methods in the class. The names of these methods usually contain terms like cached and/or array_ref. There is often another method allowing for the uncaching of such arrays, which has the effect of deleting the associated attribute from the resqpy object. The following example loads a numpy boolean array from the hdf5 file (unless it has already been cached), indicating which cells in a resqpy Grid object have geometry defined; the array is stored as an attribute of the object (cached) and also returned by the method:

faulted_grid.cell_geometry_is_defined_ref()

The Grid class also has a method which ensures that all arrays are cached:

faulted_grid.cache_all_geometry_arrays()

Note that these steps are triggered by application code calling resqpy methods. Apart from step 4, the calling code needs to keep track of which state the information for a particular object is in – resqpy itself is not generally keeping a handle on high level objects as they are instantiated.

When writing, the representation of an object typically passes through these states:

  1. Only in memory, as a resqpy object, with metadata and any array data held as attributes

  2. Metadata and any array data held as attributes of resqpy object; any array data also written to the hdf5 file

  3. The metadata is also stored in an lxml tree, in memory, in a form ready to be written to the epc file

  4. When all parts have been through the steps above, the metadata for all parts is written to the epc file from the lxml trees

Step 1 in this sequence is achieved by calling the initialization method of the resqpy class with arguments set to indicate import from a different format. Or an empty resqpy object can be instantiated and all the attributes set by the calling code. Only when the object’s attributes are fully populated can the representation proceed with the rest of the steps. The s_bend dataset, unrealistically, uses a single measured depth datum for 4 wells. Here is some example code for creating a new resqpy MdDatum object in state 1, located 5 metres to the east of the existing datum:

import resqpy.well as rqw
existing_md_uuid = model.uuid(obj_type = 'obj_MdDatum')  # we happen to know there is only one MdDatum object
existing_md_datum = rqw.MdDatum(model, uuid = existing_md_uuid)
x, y, z = existing_md_datum.location
x += 5.0
new_md_datum = rqw.MdDatum(model,
                           crs_root = existing_md_datrum.crs_root,
                           location = (x, y, z))

Step 2 is achieved by the application code calling a method, usually named write_hdf5(), for the resqpy object. As the obj_MdDatum class does not involve any array data, this step does not apply to our example.

Step 3 Each resqpy class has a method named create_xml() which generates the lxml tree representation of the metadata, in memory, and adds the part to the parent resqpy Model object, also creating relationship data structures. Here is the line for the newly created MdDatum object instantiated above:

new_md_datum.create_xml()

Step 4 is achieved by the application code calling the store_epc() method of the Model object when all objects have been prepared as far as step 3. So in the example above, when the application code has generated all the required objects, the call is simply:

model.store_epc()

At this point the data is stored persistently in the epc file (and hdf5 file) and the application can exit, or delete the model and other objects.

Temporary object states

The two situations discussed above – reading and writing – are the most common ways of working with resqpy objects. However, resqpy has been designed to support processing of models and for this a third situation can arise: the need for temporary objects. Such objects are not written to the epc file (nor their arrays to the hdf5 file) but exist only in memory as resqpy objects.

As an example of working with temporary objects, imagine an application that generates many undrilled well trajectories and then tests them against a reservoir model to select the best trajectory. The trajectories could all be saved, using the sequence for writing resqpy objects outlined above. However, perhaps there is only the need to keep the trajectory that has been selected as best. The other trajectories would be temporary.

The simplest way to work with a temporary object is simply to instantiate it. This is equivalent to step 1 of the writing sequence above. Such an object can be used for most processing purposes. Note, however, that it has not been added as a part to the nominal parent Model object, nor does any xml exist for it. Some of the resqpy method and function calls require these other steps to have been taken.

Another approach for working with temporary objects is to create a separate, temporary, Model object and to instantiate the temporary high level objects with the temporary model as the parent. The create_xml() methods of the high level objects can be called without calling the write_hdf5() methods. If the temporary model’s store_epc() method is not called, nothing will be written to the persistent file system. This is equivalent to steps 1 and 3 of the writing sequence discussed above.

Managing resqpy objects

Although a resqpy high level object is associated with a Model object (and contains a reference to the Model as an argument), the Model does not maintain a list of resqpy objects which have been instantiated for it. The Model does contain the list of RESQML parts, each of which can be used to instantiate a resqpy object (at least for the classes catered for in resqpy).

The exception is the resqpy Grid class (RESQML obj_IjkGridRepresentation), for which the Model class includes methods for optionally managing a list of resqpy Grid objects. This exception is made because grids can be memory and time intensive to instantiate, and are fundamental to all processing when working with a cellular model.

In general, though, it is up to the application code to manage the lifecycle of the resqpy objects.

RESQML to resqpy class mapping

The table below shows which high level resqpy class is used to represent each RESQML class. The blank rows indicate that a high level resqpy class has not yet been implemented for the RESQML class. (The lowest level resqpy code is generic, so steps 1 & 2 of the reading sequence above will function for all RESQML classes, as will step 4 of the writing sequence.)

RESQML class

array data

primary resqpy class

obj_Activity

obj_ActivityTemplate

obj_BlockedWellboreRepresentation

small

resqpy.well.BlockedWell

obj_BoundaryFeature

none

resqpy.organize.BoundaryFeature

obj_BoundaryFeatureInterpretation

none

resqpy.organize.BoundaryFeatureInterpretation

obj_CategoricalProperty

large

resqpy.property.PropertyCollection

obj_CategoricalPropertySeries

obj_CommentProperty

obj_CommentPropertySeries

obj_ContinuousProperty

large

resqpy.property.PropertyCollection

obj_ContinuousPropertySeries

obj_DeviationSurveyRepresentation

small

resqpy.well.DeviationSurvey

obj_DiscreteProperty

large

resqpy.property.PropertyCollection

obj_DiscretePropertySeries

obj_DoubleTableLookup

obj_EarthModelInterpretation

none

resqpy.organize.EarthModelInterpretation

obj_EpcExternalPartReference

obj_FaultInterpretation

none

resqpy.organize.FaultInterpretation

obj_FluidBoundaryFeature

none

resqpy.organize.FluidBoundaryFeature

obj_FrontierFeature

none

resqpy.organize.FrontierFeature

obj_GenericFeatureInterpretation

obj_GeneticBoundaryFeature

none

resqpy.organize.GeneticBoundaryFeature

obj_GeobodyBoundaryInterpretation

none

resqpy.organize.GeobodyBoundaryInterpretation

obj_GeobodyFeature

none

resqpy.organize.GeobodyFeature

obj_GeobodyInterpretation

none

resqpy.organize.GeobodyInterpretation

obj_GeologicUnitFeature

none

resqpy.organize.GeologicUnitFeature

obj_GeologicUnitInterpretation

none

resqpy.strata.GeologicUnitInterpretation

obj_GlobalChronostratigraphicColumn

obj_GpGridRepresentation

obj_Grid2dRepresentation

large

resqpy.surface.Mesh

obj_Grid2dSetRepresentation

obj_GridConnectionSetRepresentation

large

resqpy.fault.GridConnectionSet

obj_HorizonInterpretation

none

resqpy.organize.HorizonInterpretation

obj_IjkGridRepresentation

large

resqpy.grid.Grid

obj_LocalDepth3dCrs

none

resqpy.crs.Crs

obj_LocalGridSet

obj_LocalTime3dCrs

none

resqpy.crs.Crs

obj_MdDatum

none

resqpy.well.MdDatum

obj_NonSealedSurfaceFrameworkRepresentation

obj_OrganizationFeature

none

resqpy.organize.OrganizationFeature

obj_PlaneSetRepresentation

obj_PointSetRepresentation

large

resqpy.surface.PointSet

obj_PointsProperty

large

resqpy.property.PropertyCollection

obj_PolylineRepresentation

small

resqpy.lines.Polyline

obj_PolylineSetRepresentation

small

resqpy.lines.PolylineSet

obj_PropertyKind

none

resqpy.property.PropertyKind

obj_PropertySet

none

resqpy.property.PropertyCollection

obj_RedefinedGeometryRepresentation

obj_RepresentationIdentitySet

obj_RepresentationSetRepresentation

obj_RockFluidOrganizationInterpretation

obj_RockFluidUnitFeature

none

resqpy.organize.RockFluidUnitFeature

obj_RockFluidUnitInterpretation

obj_SealedSurfaceFrameworkRepresentation

obj_SealedVolumeFrameworkRepresentation

obj_SeismicLatticeFeature

obj_SeismicLineFeature

obj_SeismicLineSetFeature

obj_StratigraphicColumn

none

resqpy.strata.StratigraphicColumn

obj_StratigraphicColumnRankInterpretation

none

resqpy.strata.StratigraphicColumnRank

obj_StratigraphicOccurrenceInterpretation

obj_StratigraphicUnitFeature

none

resqpy.strata.StratigraphicUnitFeature

obj_StratigraphicUnitInterpretation

none

resqpy.strata.StratigraphicUnitInterpretation

obj_StreamlinesFeature

obj_StreamlinesRepresentation

obj_StringTableLookup

none

resqpy.property.StringLookup

obj_StructuralOrganizationInterpretation

obj_SubRepresentation

obj_TectonicBoundaryFeature

none

resqpy.organize.TectonicBoundaryFeature

obj_TimeSeries

none

resqpy.time_series.TimeSeries

obj_TriangulatedSetRepresentation

large

resqpy.surface.Surface

obj_TruncatedIjkGridRepresentation

obj_TruncatedUnstructuredColumnLayerGridRepresentation

obj_UnstructuredColumnLayerGridRepresentation

obj_UnstructuredGridRepresentation

large

resqpy.unstructured.UnstructuredGrid

obj_WellboreFeature

none

resqpy.organize.WellboreFeature

obj_WellboreFrameRepresentation

small

resqpy.well.WellboreFrame

obj_WellboreInterpretation

none

resqpy.organize.WellboreInterpretation

obj_WellboreMarkerFrameRepresentation

small

resqpy.well.WellboreMarkerFrame

obj_WellboreTrajectoryRepresentation

small

resqpy.well.Trajectory