tem_restart_module Module

This module provides the main IO facilities to read and write large amounts of data during the simulation.

Especially, it enables the dumping of the simulation status (e.g. the state vector) to disk and reading the information later on to restart the simulation at a certain point of time. Additionally, information like solver, version, number of elements, etc. are stored in a seperate header file.

The general procedure is as follows:

  • read the configuration from the lua file with tem_load_restart
  • initialize the read/write restart with tem_init_restart
  • read restart
  • write restart

Find more information about usage of restart in the section '[restart_usage]'

The Restart / Harvester Format

@note Todo Add an extensive description of the restart format here

The Restart Usage

To use the restart functionality the user has to define the io buffersize

 -- IO buffer size in MB (default = 8)
 io_buffer_size = 1

and the restart settings in the lua configuration file.

 -- Restart settings
 restart = { read  = 'restart/gaussian_lastHeader.lua',
                     -- file to restart from
             write = 'restart/',
                     -- prefix to write the files to
             time_control = { min = 0, max = 10, interval = 10,
                              align_trigger = false,
             }
             -- timing definitions (either iterations or simulation time)
           }

Another option to read restart is from initial conditions. The user then has to define restart data to be read into a variable in this way:

 initial_condition = { file = 'restart/channel_lastHeader.lua',
   depend = {
     variable = { {'pdf', 19} },
     usescheme = 'mini-channel'
   }
 }

If only the write option shall be used the identifier read has to be removed and vice versa.


Uses

Used by

  • module~~tem_restart_module~~UsedByGraph module~tem_restart_module tem_restart_module module~hvs_output_module hvs_output_module module~hvs_output_module->module~tem_restart_module module~tem_general_module tem_general_module module~tem_general_module->module~tem_restart_module module~tem_tracking_module tem_tracking_module module~tem_tracking_module->module~hvs_output_module

Derived Types

type, public ::  tem_restartControl_type

Control the behavior of the restart, like at which point in time etc.

Components

Type Visibility Attributes Name Initial
logical, public :: init_on_missing

Do a normal initialization if the read restart file is not found?
logical, public :: readRestart

is the restart read active?
logical, public :: writeRestart

is the restart write active?
character(len=PathLen), public :: readFileName

read the restart file from following file
character(len=PathLen), public :: writePrefix

write the restart file into a file with the following prefix
type(tem_timeControl_type), public :: timeControl

control about when to do the restart

type, public ::  tem_restartHeader_type

Define quantities like the prefix, the mesh and the timestamp

Components

Type Visibility Attributes Name Initial
character(len=LabelLen), public :: solverTag = 'unknown'

a unique tag for the solver including version
character(len=LabelLen), public :: solverKind

solver creating the input data (Ateles, Musubi)
character(len=LabelLen), public :: simName = ''

the simulation name
character(len=pathLen), public :: solverConfigFile = ''

the solver config file name
character(len=PathLen), public :: binName

name of the binary files (this variable is set for every timestamp differently, used for read and write)
character(len=PathLen), public :: binPrefix

prefix for the binary files (includes directory and file prefix)
character(len=PathLen), public :: headerPrefix

prefix for the header files (includes directory and file prefix)
character(len=16), public :: timestamp

the current time generated from tem_time_module
integer, public :: nElems

mesh directory Number of elements as read from the restart header
type(tem_varSys_type), public :: varSys

variable system dumped in restart header

type, public ::  tem_restart_type

The restart type defining everything related to the disk input/output

Components

Type Visibility Attributes Name Initial
type(tem_comm_env_type), public :: comm

communicator for the processes participating in this restart (might be only a subset of the global communicator)
integer, public :: nChunkElems

actual number of elements in the current chunk (= chunkSize or
type(tem_file_layout_type), public :: read_file

Description of the data layout to use when reading a file.
type(tem_file_layout_type), public :: write_file

Description of the data layout to use when writing a file.
type(tem_restartControl_type), public :: controller

Control the behavior of the restart, like at which point in time etc.
type(tem_restartHeader_type), public :: header

Define quantities like the prefix, the mesh and the timestamp
integer, public :: binaryUnit

unit integer to write binary data to
type(tem_varMap_type), public :: varMap

name and position of variables in global variable system

Read more…
integer, public :: nScalars
integer, public :: solSpec_unit = -1

scratch file unit contains solver specific info in dump in restart header This file should contain the information in form of a Lua script.
type(tem_time_type), public :: lastWritten

The time when the last restart file was written.

type, private ::  tem_file_layout_type

Components

Type Visibility Attributes Name Initial
integer, public :: nDofs = 1

Number of degrees of freedom for each of the scalars entries in the varibale system in the file.
integer, public :: nChunks

Number of local chunks required to fit complete data in. Set in routine: tem_restart_getTotalChunks
integer, public :: maxnChunks

Globally maximal number of chunks required. Set in routine: tem_restart_getTotalChunks
integer, public :: chunkSize

Number of elements that fit into the buffer. Set in routine: tem_restart_getTotalChunks
integer, public :: ftype

Handle for the MPI type to describe the view for IO in binary IO
integer, public :: vectype

Handle for the MPI type describing the vector of data in each element
integer(kind=MPI_OFFSET_KIND), public :: displacement

start of view for MPI_SET_VIEW

Subroutines

public subroutine tem_load_restart(me, conf, tree, timing, globProc, parent_table, key)

Read all necessary information for the restart from the lua config file.

Read more…

Arguments

Type IntentOptional Attributes Name
type(tem_restart_type), intent(inout) :: me

restart type to be filled
type(flu_State) :: conf

lua configuration file
type(treelmesh_type), intent(inout) :: tree

mesh, provided in treelm format
type(tem_time_type), intent(inout) :: timing

the timing for re-setting the times
type(tem_comm_env_type), intent(in) :: globProc

Global process communicator env
integer, intent(in), optional :: parent_table

optional parent handle
character(len=*), intent(in), optional :: key

optional key for table

public subroutine tem_init_restart(me, solver, varMap, tree, subTree, nDofs_write, chunkSize, solSpec_unit)

Initialize the global restart data type and prepare for the restart output

Read more…

Arguments

Type IntentOptional Attributes Name
type(tem_restart_type), intent(inout) :: me

The restart object to initialize.
type(tem_solveHead_type), intent(in), optional :: solver

Details about the solver.
type(tem_varMap_type), intent(in) :: varMap

Description of each variable system. This is ignored, if the data is provided by reading a restart. Contains position of variables to dump in restart file in global variable system for a scheme
type(treelmesh_type), intent(in) :: tree

Mesh, provided in treelm format
type(tem_subTree_type), intent(in), optional :: subTree

optional subTree of the given tree
integer, intent(in), optional :: nDofs_write

number of degrees of freedom for each variable of the equation system
integer, intent(in), optional :: chunkSize

use predefined chunkSize
integer, intent(in), optional :: solSpec_unit

Solver specific unit for restart header

public subroutine tem_init_restart_alloc(me, comm, rank, comm_size, solver, nDofs_write)

Author
Kartik Jain

This subroutine initializes the restart in case of reading initial conditions from restart file. The dependent scheme id is passed as input.

Arguments

Type IntentOptional Attributes Name
type(tem_restart_type), intent(inout) :: me
integer, intent(in) :: comm
integer, intent(in) :: rank
integer, intent(in) :: comm_size
type(tem_solveHead_type), intent(in), optional :: solver
integer, intent(in), optional :: nDofs_write

public subroutine tem_init_restart_create_types(me, elemOff, locElems)

This subroutine creates MPI types for reading the restart.

Arguments

Type IntentOptional Attributes Name
type(tem_restart_type), intent(inout) :: me
integer(kind=long_k), intent(in) :: elemOff
integer, intent(in) :: locElems

public subroutine tem_restart_dump_data(restart, varsys, tree, time, subtree)

This subroutine evaluated get_element and dump each chunk

Read more…

Arguments

Type IntentOptional Attributes Name
type(tem_restart_type), intent(inout) :: restart

The restart object describing how and what to output.
type(tem_varSys_type), intent(in) :: varsys

Description of the available variable system to get the given varnames from.
type(treelmesh_type), intent(in) :: tree

Mesh to write the data on.
type(tem_time_type), intent(in) :: time

Point in time to use for this data.

Read more…
type(tem_subTree_type), intent(in), optional :: subtree

Optional restriction of the elements to output.

public subroutine tem_restart_writeData(restart, chunk)

This subroutine dumps the given chunk to a given position in the restart file.

Arguments

Type IntentOptional Attributes Name
type(tem_restart_type), intent(inout) :: restart

The restart object describing how and what to output.
real(kind=rk), intent(in) :: chunk(:)

The data to output. It is organized as a serialized array of all scalar entries of all variable systems. Where first all the data for the nElems of the first variable system is provided. Within each variable system the data is organized elementwise.

public subroutine tem_restart_readData(restart, chunk)

Read data from a restart file.

Arguments

Type IntentOptional Attributes Name
type(tem_restart_type), intent(inout) :: restart

Restart description to read the data from
real(kind=rk), intent(out) :: chunk(:)

Chunk of memory to put the read data into

public subroutine tem_restart_readData_single(restart, chunk, offset)

subroutine which reads data from restart file corresponding to the input variable system

Arguments

Type IntentOptional Attributes Name
type(tem_restart_type), intent(inout) :: restart

Restart object to read the data from
real(kind=rk), intent(out) :: chunk(:)

Memory to chunk to put the data into
integer, intent(in) :: offset

Offset of the chunk of data to get in the global data

public subroutine tem_restart_openRead(me)

open the restart dump file and write out the 'normal' restart header.

Arguments

Type IntentOptional Attributes Name
type(tem_restart_type) :: me

the restart information

public subroutine tem_restart_openRead_single(me)

Open the restart dump file and set file view for the input variable system

Arguments

Type IntentOptional Attributes Name
type(tem_restart_type) :: me

the restart information

public subroutine tem_restart_closeRead(me)

This subroutine closes the restart dump file and writes the last header.

Arguments

Type IntentOptional Attributes Name
type(tem_restart_type) :: me

the restart information

public subroutine tem_restart_closeRead_single(me)

Close the restart dump file corresponding to a particular variable system

Arguments

Type IntentOptional Attributes Name
type(tem_restart_type) :: me

the restart infotmation

public subroutine tem_restart_openWrite(me, tree, timing, varSys, subTree, label, suffix)

open the restart dump file and write out the 'normal' restart header as well as the mesh.

Arguments

Type IntentOptional Attributes Name
type(tem_restart_type) :: me

the restart infotmation
type(treelmesh_type) :: tree

mesh, provided in treelm format
type(tem_time_type), intent(in) :: timing

current simulation time information
type(tem_varSys_type), intent(in) :: varSys

the used var systeme
type(tem_subTree_type), intent(inout), optional :: subTree

optional subTree of the given tree
character(len=*), intent(in), optional :: label

additional label for the filename (needed for tracking in harvester format)
character(len=*), intent(in), optional :: suffix

optional suffix (if present NO timestamp will be added!!!!)

public subroutine tem_restart_writeHeader(me, tree, varSys, subTree, timing, lastHeader, suffix)

This subroutine closes the restart dump file and writes the last header.

Arguments

Type IntentOptional Attributes Name
type(tem_restart_type) :: me

the restart header info
type(treelmesh_type) :: tree

mesh, provided in treelm format
type(tem_varSys_type), intent(in) :: varSys

global variable system defined in solver
type(tem_subTree_type), intent(in), optional :: subTree

optional subTree of the given tree
type(tem_time_type), intent(in) :: timing
logical, optional :: lastHeader

is this header a last header
character(len=*), intent(in), optional :: suffix

optional suffix (if present NO timestamp will be added!!!!)

public subroutine tem_restart_closeWrite(me, tree, timing, varSys, subTree)

Complete a number of empty writes (due to higher amount of mpi_file_writes from other processes to finalize the write process), close the restart dump file and write the last header.

Arguments

Type IntentOptional Attributes Name
type(tem_restart_type) :: me

the restart information
type(treelmesh_type) :: tree

mesh, provided in treelm format
type(tem_time_type), intent(in) :: timing

The timing object holding the simulation time
type(tem_varSys_type), intent(in) :: varSys

global variable system defined in solver
type(tem_subTree_type), optional :: subTree

optional subTree of given tree

public subroutine tem_restart_getTotalChunks(restart, nElems, comm, chunkSize)

calculate the maximum number of elements which fit into the output buffer = chunk

Read more…

Arguments

Type IntentOptional Attributes Name
type(tem_restart_type), intent(inout) :: restart

the restart type
integer, intent(in) :: nElems

mesh, provided in treelm format optional subTree
integer, intent(in) :: comm

mesh, provided in treelm format optional subTree
integer, intent(in), optional :: chunkSize

optional predefined chunksize

public subroutine tem_restart_finalize(me)

Finalizing a restart object

Arguments

Type IntentOptional Attributes Name
type(tem_restart_type), intent(inout) :: me

the restart type to close

private subroutine tem_restart_readHeader(me, timing, globProc, tree)

read the restart header lua file and hand the information to the required data types, re-set the time ...

Arguments

Type IntentOptional Attributes Name
type(tem_restart_type) :: me

the restart header info
type(tem_time_type), intent(inout) :: timing

the timing for re-setting the times
type(tem_comm_env_type), intent(in) :: globProc

Global communicator
type(treelmesh_type), intent(inout) :: tree

mesh, provided in treelm format