Saving Or Deleting CPDN Result Data Files

From Unofficial BOINC Wiki

Contents 1 General 2 Preconditions 3 Stage #1 - Decide on policy 4 Stage #2 - Identification of Completed Models 4.1 Introduction 4.2 Step #2a - "Drill-Down" To The Climateprediction.net Models 4.3 Step #2b - Identify Completed Models 5 Stage #3 - Deleting CPDN Result Data Files 5.1 Introduction 5.2 Step #3a - Deleting The Result Data Files

[edit] General

The Science Application for the Climateprediction.net Project will finish processing the Work Unit Data File into several Result Data Files. Of these files, about 7 Mega-Bytes will be uploaded to the Project's Data Server with another large set of files that contains the details of the data processing. This collection of the Result Data Files can be 330 Mega-Bytes or more of information.

In most cases, the Climateprediction.net Project will not need these files. However, there are cases where they would like to do additional processing of the model you processed. If you save these files, well, obviously it would be faster than if you have deleted the files. Now we have a situation. You may not have a lot of disk space to store these files. So, in this case, we want to move them to off-line storage.

[edit] Preconditions

In order to complete the procedure outlined in this guide, you must meet the following preconditions:

• The computer must be "Attached" to the Climateprediction.net Project.
• The computer must have processed at least one model to completion.

Note:

Running a model to completion means either to a "Success", or "Client Error" Outcome. In general, a model that does not complete a substantial portion of the total processing time will not be of much worth, trouble or space. In those cases, we suggest a simple deletion policy once the Result shows in the Web Site as a "Client Error" Outcome.

[edit] Stage #1 - Decide on policy

Each Work Unit can leave behind up to 330Mb of data or 1GB for Sulphur Cycle model.

Only a small part (7MB or 20MB) of the data has been uploaded. The rest could still be useful. Obviously the most useful/wanted information has been selected.

The data is left behind because different people will have different policies regarding the Climateprediction.net data.

Interested Keepers Some people will want to keep it on their hard disks so they can look at the model with CPView or the advanced visualisation program.

Archivers Some people are willing to store data if there is a chance that having it available will save the work being recrunched. The suggestion is that it might prove useful at almost any time in the next 10 years. Most use of the data will probably peak in about 2 years then gradually tail off. Moving it to a CD or DVD is fine - the CP team cannot upload it from your PC without getting in touch with you. Even if some of these CDs/DVD get lost before 10 years is up, at least in some cases it saves the model having to be recrunched.

Deleters Others will decide that they are 'NOT A FREE STORAGE FACILITY'. You are free to decide to delete the data. It is not disasterous, if it later turns out the CP team want the information, the WU can be handed out again to be crunched. Some people will prefer that this should be avoided where possible but it is far better to have people who crunch and delete the data than not have such crunchers in the first place.

Some people have been asked to upload specific runs, so it is not ridiculously unlikely that they will ask for more to be uploaded. Though I don't think it is a large number so far.

Deciding On a Delete/Archive Policy If a model has crashed during the first Phase, there is very little point in keeping it. There could be some point in keeping a model that got further as the scientists are interested in finding out how models are crashing. Your policy will depend on how willing you are to save data just in case it may be useful.

Uploads: There is a facility for uploading classic runs. I don't believe this will take BOINC models. Climateprediction.net don't have the staff or the storage capacity to cope with people sending in all the information from large numbers of BOINC runs.

[edit] Stage #2 - Identification of Completed Models

[edit] Introduction

Our first task is to locate those models that have been completed on this computer.

[edit] Step #2a - "Drill-Down" To The Climateprediction.net Models

Navigate to the BOINC Directory, which on Microsoft Windows® will usually be under the "Program Files" directory.

Open the "Program Files" directory.

Open the "BOINC Directory".

Open the "Projects" directory.

Open the "climateprediction.net" directory.

[edit] Step #2b - Identify Completed Models

within the "climateprediction.net" directory you can see I have a total of five different models (Work Units) for the Climateprediction.net Project. The Work Units include:

• 0ki3_100046863
• 3j5f_200186443
• 16rj_200075999
• 26ce_300122566
• 40fd_100209054

Now we must look to see which of these is a model we can archive and which is still in some stage of processing.

When the "0ki3_100046863" directory is opened we see that it is a model that is in progress and it only contains files that contain state information and the Work Unit Data Files.

When the "3j5f_200186443" directory is opened we see that it contains a model that is complete!

We know this because the Result Data Files are present as zip archives. We know this is a full model because there are 367 files and contains 330MB. This is therefore a candidate for archiving. Had the directory contained zip files but less than the full complement of 367 files totaling 330MB, then depending on your policy, it may be identified as for deletion without archiving it.

With a candidate identified we can now move these files off the machine onto an archive directory or onto CD.

Because everyone has their own CD burning program, I will not cover that mechanism. Also, there is the choice to delete the Result Data Files which is covered in "Stage #3 - Deleting CPDN Result Data Files"

I select a file and use the copy process to move it to the disk I use to archive important material. In my case it is a small 300 Giga-Byte RAID-5 Array. I use copy because if there is a problem with the process I still have the original files available. Yes, I am that cautious ... Aren't you?

(If you are not that cautious or have backups of your BOINC directory, you may be willing to cut and paste the directory which is much faster.)

During the copy you see the progress of the copy.

And obviously you can copy more than one at a time.

[edit] Stage #3 - Deleting CPDN Result Data Files

[edit] Introduction

Regardless of moving the file off to CD, external storage, floppies, etc.; we still need to clean up the directories that contain the model's Result Data Files.

[edit] Step #3a - Deleting The Result Data Files

Once the directories have been moved or archived, now we need to select those directories that contain models that are no longer needed and delete them.

Note:

Obviously you want to be careful that you do not delete a model that is still being processed.

Be careful do select only those models that have completed processing.