Files and File References

From Unofficial BOINC Wiki

Contents 1 General 2 File References 3 File Management 4 UCB Source 5 Copyright ©

[edit] General

The BOINC storage model is based on files. Examples of files:

• The inputs and outputs of applications;
• Application executables, libraries, etc.

The BOINC Daemon transfers files to and from project-operated Data Servers using HTTP.

A file is described by an XML element of the form:

<file_info>
    <name>foobar</name>
    <url>http://a.b.c/foobar</url>
    <url>http://x.y.z/foobar</url>
    ...
    <md5_cksum>123123123123</md5_cksum>
    <nbytes>134423</nbytes>
    <max_nbytes>200000</max_nbytes>
    <status>1</status>
    [ <generated_locally/> ]
    [ <executable/> ]
    [ <upload_when_present/> ]
    [ <sticky/> ]
    [ <signature_required/> ]
    [ <no_delete/> ]
    [ <report_on_rpc/> ]
</file_info>

The elements are as follows:

name	The file's name, which must be unique within the project.
url	a URL where the file is (or will be) located on a data server.
md5_cksum	The MD5 checksum of the file.
nbytes	the size of the file in bytes (may be greater than 2^32).
max_nbytes	The maximum allowable size of the file in bytes (may be greater than 2^32). This is used to prevent flooding Data Servers with bogus data.
status	0 if the file is not present, 1 if the file is present, or a negative error code if there was a problem in downloading or generating the file.
generated_locally	If present, indicates that the file will be generated by an application on the Participant's Host, as opposed to being downloaded.
executable	If present, indicates that the file protections should be set to allow execution.
upload_when_present	If present, indicates that the file should be uploaded when the Science Application that generates it exits.
sticky	If present, indicates that the file should be retained on the Participant's Host after its initial use.
signature_required	If present, indicates that the file should be verified with an RSA Signature. This generally only applies to executable files.
no_delete	If present for an Work Unit Data File, indicates that the file should NOT be removed from the Data Server's Download Directory when the Work Unit is completed. Use this if a particular input file or files are used by more than one Work Unit, or will be used by future Work Units.
no_delete	If present for an output (result) file, indicates that the file should NOT be removed from the Data Server's Upload Directory when the corresponding Work Unit is completed. Use with caution - this may cause your upload directory to overflow.
report_on_rpc	Include a description of this file in scheduler RPC requests, so that the scheduler may send appropriate work using Locality Scheduling.

Once a file is created (on a data server or a participant host) it is immutable. This means that all replicas of that file are assumed to be identical.

[edit] File References

Files may be associated with Work Units, Results and Application Versions. Each such association is represented by an XML element of the form:

<file_ref>
    <file_name>foobar</file_name>
    [ <open_name>input</open_name> ]
    [ <main_program/> ]
</file_ref>

The elements are as follows:

file_name	Specifies a file.
open_name	The name by which the application will refer to the file.
main_program	Used for files associated with application versions. It indicates that this file is the application's main program.

[edit] File Management

BOINC's default behavior is to delete files around when they aren't needed any more. Specifically:

• On the Participant's Host, input files are deleted when no Work Unit refers to them, and output files are deleted when no result refers to them. Application-version files are deleted when they are referenced only from superseded Application Versions.
• On the Participant's Host, the 'sticky' flag overrides the above mechanisms and suppresses the deletion of the file. The file may deleted by an explicit Scheduling Server Delete Request. The file may also be deleted at any time by the BOINC Daemon in order to honor limits on disk-space usage.
• On the Servers, the File Deleter Daemon deletes Work Unit Data Files and Result Data Files that are no longer needed. This can be suppressed using the 'no_delete' flag, or using command-line options to the File Deleter.

[edit] UCB Source

• Storage

[edit] Copyright ©

• 2005 University of California
• 2005 Paul D. Buck

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.