wiki:DSC/zf_validation

Context Navigation

Version 11 (modified by seskar, 10 years ago) ( diff )
—

Image Validation Procedure (Preliminary Torunament)

Decide on the image name (e.g. team-image-name.ndz) and store it in /.orbit-image
Run prepare.sh and let the node shut down

Run "omf save" and save the image

      omf save -n node2-1.grid.obit-lab.org

ssh into repository1 and rename the image

  ssh repository1 
  cd /export/omf/omf-images
  mv expf-id-based-image-name.ndz team-image-name.ndz

go back to the grid console and load the image onto one pair of match nodes, i.e.:
```
  omf load -i team-image-name.ndz -t system:topo:dsc-teamA
```

turn the imaged nodes on

  omf tell -a on -t system:topo:dsc-teamA

Execute a competitive match and receive at least one packet successfully
```
  omf exec system:exp:dsc-match -- --mode COMP
```
Execute a cooperative match and receive at least one packet successfully
```
  omf exec system:exp:dsc-match -- --mode COOP
```
Do not alter the image in any way (including renaming it). The image you want to used in the final competition must exactly match the one that you validate and tell us to use. Alteration of anything (filename, changing code slightly) will yield a hash that will not match our records!
If you have multiple images that were scored, tell the ORBIT team which one you want to be used, otherwise the last one that was successfully verified for your team will be used.

Image Validation Procedure (Final Tournament)

Please note that, in order for a submitted image to be accepted for the final tournament, it has to be validated for BOTH competitive and cooperative matches

Decide on the image name (e.g. team-image-name.ndz) and store it in /.orbit-image
Run prepare.sh and let the node shut down

Run "omf save" and save the image

      omf save -n node2-1.grid.obit-lab.org

ssh into repository1 and rename the image

  ssh repository1 
  cd /export/omf/omf-images
  mv expf-id-based-image-name.ndz team-image-name.ndz

Validate With Basic Script

Validate Competitive Match

On the grid console load the saved image into one pair of the competitive match nodes and load one of the bots into the other pair, i.e.:
```
  omf load -i team-image-name.ndz -t system:topo:dsc-teamXp
  omf load -i dsc-bot1.ndz -t system:topo:dsc-teamYp
```
Execute a competitive match and receive at least one packet successfully (note the 20 second match duration option)
```
  omf exec system:exp:dsc-match -- --mode COMP --arena PRI --runtime 20
```

Validate Cooperative Match

On the grid console load the saved image into one pair of cooperative match nodes and load two other bots into the other two sets of cooperative match nodes i.e.:

  omf load -i team-image-name.ndz -t system:topo:dsc-teamAp
  omf load -i dsc-bot6.ndz -t system:topo:dsc-teamBp
  omf load -i dsc-bot5.ndz -t system:topo:dsc-teamCp

Execute a cooperative match and receive at least one packet successfully

  omf exec system:exp:dsc-match -- --mode COOP --arena PRI --runtime 20

After each successful validation, you should receive a confirmation email with the score. Note that you can validate multiple images.
Do not alter the image in any way (including renaming it) once you have received validation emails for the image you want to be used in the final competition.
Fill in the submission form with the image name that was successfully validated (you have to choose one image out of the set that passed the validation procedure) and submit it by email.

Validate With Tournament Scripts

To prepare the extended framework create the three configuration files in current working directory:

teams.yaml

comp_matches.yaml

coop_matches.yaml

T0:
  name: Team Name
  image: team-image-name.ndz
T1:
  name: ORBIT team 1
  image: dsc-bot1.ndz
T2:
  name: ORBIT team 2
  image: dsc-bot2.ndz 
T3:
  name: ORBIT team 3
  image: dsc-bot3.ndz
T4:
  name: ORBIT team 1
  image: dsc-bot4.ndz
T5:
  name: ORBIT team 2
  image: dsc-bot5.ndz
T6:
  name: ORBIT team 3
  image: dsc-bot6.ndz

G1:
   1: T0
   2: T2

G1:
   1: T0
   2: T6
   3: T5

For competitive validation, in the same directory execute:
```
  dsc-comp 1 A
```
For cooperative validation, in the same directory execute:
```
  dsc-coop 1 
```
After every successful validation, you should receive confirmation email with the score. Note that you can validate multiple images.
Do not alter the image in any way (including renaming it) once you have received validation emails for the image you want to use for the final competition.
Fill in the submission form with the image name that was successfully validated (you have to choose one image out of the set that passed the validation procedure) and submit it over email.
We strongly encourage teams to play at least one match in all 6 possible arenas (each run takes between 4 and 8 minutes depending on the image sizes and match duration) i.e. to exercise your image in all 6 arenas with 20 second matches run:
```
  /usr/local/bin/dsc-comp -r 20 1 A
  /usr/local/bin/dsc-comp -r 20 1 B
  /usr/local/bin/dsc-comp -r 20 -s 1 A
  /usr/local/bin/dsc-comp -r 20 -s 1 B
  /usr/local/bin/dsc-coop -r 20 1
  /usr/local/bin/dsc-coop -r 20 -s 1
```
If time is absolutely critical, you can try it in parallel e.g. in the same directory, with two ssh sessions, execute:

ssh session 1

ssh session 2

/usr/local/bin/dsc-comp -e load 1 A
/usr/local/bin/dsc-comp -e match -r 20 1 A
/usr/local/bin/dsc-comp -e load 1 B
/usr/local/bin/dsc-comp -e match -r 20 1 B
/usr/local/bin/dsc-coop -e load 1 
/usr/local/bin/dsc-coop -e match -r 20 1
-

-
/usr/local/bin/dsc-comp -e load -s 1 A
/usr/local/bin/dsc-comp -e match -s -r 20 1 A
/usr/local/bin/dsc-comp -e load -s 1 B
/usr/local/bin/dsc-comp -e match -s -r 20 1 B
/usr/local/bin/dsc-coop -e load -s 1 
/usr/local/bin/dsc-coop -e match -s -r 20 1

Note: See TracWiki for help on using the wiki.

Download in other formats:

Plain Text