BatchMake: Difference between revisions

From KitwarePublic
Jump to navigationJump to search
Line 261: Line 261:




===ListDirInDir List all the directories in a specified directory===
===ListDirInDir - List all the directories in a specified directory===
'''ListDirInDir( <dirlist> <directory> [ *.* ] )'''
'''ListDirInDir( <dirlist> <directory> [ *.* ] )'''


Line 274: Line 274:
           number3
           number3


set(dir F:\Example)
set(dir F:\Example)
ListDirInDir(example ${dir})
ListDirInDir(example ${dir})
echo(${example})
echo(${example})
 
This script will print the following lines on the screen:
number1/ number2/ number3/
 
===ListFileInDir - List all the files in a specified directory===
'''ListFileInDir( <filelist> <directory> [ *.* ] )'''
 
<filelist> contains the name of all the files included in <directory>. We can also list only the files with a specific string in their names : for instance if *.txt' is added at the end of ListFileInDir only the files wtih the '.txt' extension will be listed.
 
'''Example :'''
Structure of the Hard Drive :
  F:\
    Example
        number1.txt
        number2.txt
        number3.txt
 
set(dir F:\Example)
ListFileInDir(example ${dir})
echo(${example})
 
This script will print the following lines on the screen:
number1.txt number2.txt number3.txt
 
 
===OpenTCPSocket - Open a TCP/IP socket===
'''OpenTCPSocket( <SocketName> <IPAddress> <Port> )'''
 
'''Example :'''
OpenTCPSocket(MySocket,10.0.0.1,8081)
 
 
===Randomize - Create a random number===
'''Randomize( <variable> <uniform | gaussian | categorical> <min> <max> )'''
 
<variable> contains the result. The distribtion can be set thanks to one of these 3 words : uniform, gaussian or categorical. The range an be set thanks to <min> and <max>.
 
'''Example :'''
Randomize(result uniform 0 1)
echo(${result})
 
This script will print on the screen a random number between 0 and 1 following a uniform distribution.
 
 
===Run - Execute a program in a thread===
'''Run( <output variable> <'program.exe param1 param2 ....'> )'''
 
The program called 'program.exe' will be launched with the parameters 'param1 param2' ...
Note that the Application Wrapper can be used to defined the parameters as names.
 
'''Example :'''
Run( output 'MyProgram.exe -in input -out output' )
 
This script will run the program called MyProgram.exe with the parameters input and output.


= Distributed Computing =
= Distributed Computing =


= BatchBoards =
= BatchBoards =

Revision as of 19:56, 9 October 2007

BatchMake is a cross platform tool for batch processing of large amount of data. BatchMake can process datasets locally or on distributed systems using Condor.

Common Features:

  • BSD License
  • CMake like scripting langage
  • Distributed Scripting via Condor
  • Central remote website for online statistical analysis
  • User Interface using FLTK
  • Cross platform

BatchMake

List of available commands:

  • AppendFile Add a string to the end of a file
  • CloseTCPSocket Close a TCP/IP socket
  • DashboardExperiment Specify the encrypted name of the experiment to report on
  • DashboardHost Specify the hostname of the central database manager
  • DashboardNotify Specify the hostname of the central database manager
  • DashboardSend Send the results of a program to the central database for data collection
  • DeleteFile Delete a file on disk
  • Echo Print messages on the standard output
  • ExtractSlice Extract a slice from a 3D image volume
  • ExtractString Extract a string from a string chain
  • ForEach Create a "for" loop
  • GetParam Access directly a parameter from a sequence
  • If Defines a conditional statement
  • Inc Increment a number by a certain amount
  • Include Include another batchmake script
  • Int Convert a variable to integer
  • ListDirInDir List all the directories in a specified directory
  • ListFileInDir List all the files in a specified directory
  • OpenTCPSocket Open a TCP/IP socket
  • Randomize Create a random number
  • Run Execute a program in a thread
  • Sequence Create a sequence of numbers
  • Set Assign a value to a variable
  • SetApp Assign a variable to an Application defined by the Application Wrapper
  • SetAppOption Assign a value to a predefined option of a program
  • SendTCP Send a message via TCP socket
  • Sin Evaluate the sin() function
  • WriteFile Write variables into a file

How to use each command:

AppendFile - Add a string to the end of a file

AppendFile( <filename> <value1> <value2> ... )

The first parameter is the name of the file and all the others parameters will be added at the end of this file.

Example :

sequence(seq 1 3 1)
foreach(value ${seq})
appendFile('C:/bmtest.txt' 'value: ${value}\n')
endforeach(value)

This script will add the following strings at the end of the file entitled "C:/bmtest.txt" :

"value: 1"
"value: 2"
"value: 3"

CloseTCPSocket - Close a TCP/IP socket

CloseTCPSocket( <SocketName> )

Example :

CloseTCPSocket(MySocket)


DashboardExperiment - Specify the encrypted name of the experiment to report on

DashboardExperiment( <encryptedExperiment> )

The encryptedExperiment is given when creating a new experiment online (this requires a login/pass)

Example :

DashboardExperiment( 7YF04vELWW4m1B88t)


DashboardHost - Specify the hostname of the central database manager

DashboardHost( <hostname> )

Example :

DashboardHost(www.kitware.com)

DashboardNotify - Add a new string to the central database

DashboardNotify( <message> )

Example :

DashboardNotify('End of first experiment')


DashboardSend - Send the results of a program to the central database for data collection

DashboardSend( <methodid> <value1> <value2> )

Example :

DashboardSend(1,${value1},${value2})


DeleteFile - Delete a file on disk

DeleteFile( <filename> )

Example :

DeleteFile('myfile.txt')


Echo - Print messages on the standard output

Echo( <value1> <value2> ... )

Example :

sequence(seq 1 2 1)
sequence(seq2 1 2 1)
foreach(value ${seq})
foreach(value2 ${seq2})
Echo(value: ${value} ${value2})
endforeach(value2)
endforeach(value)

This script will print the following lines on the screen :

value: 1 1
value: 1 2
value: 2 1
value: 2 2


ExtractSlice - Extract a slice from a 3D image volume

ExtractSlice( <3D image filename> <2D image filename> <orientation> <slice> )

Example :

ExtractSlice('head.mha','slice.jpg',0,23)

This command extracts the saggital slice #23 from the volume head.mha and write the extracted slice to slice.jpg


ExtractString - Extract a string from a string chain

ExtractString( <output> <input> <lenght> [FROMEND] [KEEPEND] )

The parameter <output> will contain the <lenght> first characters of <input> if FROMEND is not set. With FROMEND the <lenght> last characters won't be copied. if KEEPEND is specified then the end of the string is returned.

Example :

set(example 'hello')
extractstring(result ${example} 1)
extractstring(result2 ${example} 2 FROMEND)
extractstring(result3 ${example} 2 KEEPEND)
extractstring(result4 ${example} 2 FROMEND KEEPEND) 
echo(${result})
echo(${result2})
echo(${result3})
echo(${result4})

This script will print the following lines on the screen :

h
hel
llo
lo


ForEach / EndForEach - Create a "for" loop

ForEach( <variable> <sequence> )

EndForEach( <variable> )

During the ith iteration, <variable> contains the ith parameter of <sequence>. The EndForEach command allows to define the end of the loop.

Example :

set(seq a b c d e f)
ForEach(letter ${seq})
echo(${letter})
EndForEach(letter)

This script will print the following lines on the screen :

a
b
c
d
e
f


GetParam - Access directly a parameter from a sequence

GetParam( <new variable> <sequence> <number> [<number2>] ... )

<number> corresponds to the argument of <sequence> to copy into <new variable>. We can copy as many arguments of <sequence> as we want adding at the end the rank of the argument in <sequence>.

Example :

sequence(seq 1 7 2)
echo(${seq})
GetParam(new1 ${seq} 0)
GetParam(new2 ${seq} 3)
echo(${new1})
echo(${new2})
GetParam(new ${seq} 0 2)
echo(${new})

This script will print the following lines on the screen :

1 3 5 7
1
7
1 5


If / EndIf - Defines a conditional statement=

If( <variable1> == or <= or >= or != <variable2> ) EndIf( <variable> )

Example :

set(var1 2)
set(var2 4)
If(${var1} <= ${var2})
echo(${var1} <= ${var2})
EndIf(${var1})

set(char1 test)
set(char2 test)
If(${char1} == ${char2})
echo(${char1} == ${char2})
EndIf(${char1})

This script will print the following lines on the screen :

2 <= 4
test == test 


Inc Increment a number by a certain amount

Inc( <variable> <number> )

<variable> = <variable> + <number>

Example :

Set(value 2)
Inc(${value} 5)
echo(2 + 5 = ${value})

This script will print the following lines on the screen :

2 + 5 = 7.000000 


Include - Include another batchmake script

Include( <ScriptFilename> )

Example :

#Script1 : F:\Script1.bms
echo('Script1')

#Script2 : F:\Script2.bms
echo('Script2')

#Script3 :
Include(F:\Script1.bms)
Include(F:\Script2.bms)

When we execute Script3 the following lines will be printed on the screen :

Script1
Script2

Int - Convert a variable to integer

Int( <variable> )

Example :

Set(value 2)
Inc(${value} 5)
Int(${value})
echo(2 + 5 = ${value})

This script prints the following lines on the screen:

2 + 5 = 7  ( instead of 7.000000 if we Int() is not used)


ListDirInDir - List all the directories in a specified directory

ListDirInDir( <dirlist> <directory> [ *.* ] )

<dirlist> contains the name of all the subdirectories included in <directory>. We can also list only the subdirectories with a specific string in their names : for instance if 'doc*' is added at the end of ListDirInDir only the subdirectories started wtih 'doc' will be listed.

Example :

Structure of the Hard Drive :
 F:\
    Example
         number1
         number2
         number3
set(dir F:\Example)
ListDirInDir(example ${dir})
echo(${example})

This script will print the following lines on the screen:

number1/ number2/ number3/

ListFileInDir - List all the files in a specified directory

ListFileInDir( <filelist> <directory> [ *.* ] )

<filelist> contains the name of all the files included in <directory>. We can also list only the files with a specific string in their names : for instance if *.txt' is added at the end of ListFileInDir only the files wtih the '.txt' extension will be listed.

Example :

Structure of the Hard Drive :
 F:\
    Example
        number1.txt
        number2.txt
        number3.txt 
set(dir F:\Example)
ListFileInDir(example ${dir})
echo(${example})

This script will print the following lines on the screen:

number1.txt number2.txt number3.txt


OpenTCPSocket - Open a TCP/IP socket

OpenTCPSocket( <SocketName> <IPAddress> <Port> )

Example :

OpenTCPSocket(MySocket,10.0.0.1,8081)


Randomize - Create a random number

Randomize( <variable> <uniform | gaussian | categorical> <min> <max> )

<variable> contains the result. The distribtion can be set thanks to one of these 3 words : uniform, gaussian or categorical. The range an be set thanks to <min> and <max>.

Example :

Randomize(result uniform 0 1)
echo(${result}) 

This script will print on the screen a random number between 0 and 1 following a uniform distribution.


Run - Execute a program in a thread

Run( <output variable> <'program.exe param1 param2 ....'> )

The program called 'program.exe' will be launched with the parameters 'param1 param2' ... Note that the Application Wrapper can be used to defined the parameters as names.

Example :

Run( output 'MyProgram.exe -in input -out output' )

This script will run the program called MyProgram.exe with the parameters input and output.

Distributed Computing

BatchBoards