PLI Cyto Style Transfer

Repository for PLI - Cyto style transfer for master thesis of Ragib

Project structure

Code

cli/: All command line scripts belong here. All scripts should be written such that they can be executed from the root of the project, and are only using relative paths for portability.

lab/: All experimental code (notebooks etc.) belongs here. Once runnable it should be converted to cli programs in the folder cli/ for a more flexible use.

code/: Shared code between lab/ and cli/ but also between projects belongs here.

env/: This folder links to the PLI JSC Environment as submodule. The requirement files serve as a guideline for instantiating local Python environments. Adjustments can be made by either creating a new branch or detaching the submodule from the project.

Data

in/: Contains links to datasets, raw data or data that were processed by other projects and are used as input for this project.

data/: Larger data that is being produced or stored along this project (e.g. trained models, processed data, ...) goes here. Files here are annexed and version controlled via DataLad.

doc/: All kinds of documentation such as charts, tables, graphics or conclusions that should persist belong here.

tmp/: Project specific temporary files that do not need to be shared across siblings of this project go here. Rule of thumb: "Everything that can safely be deleted to save disk space at the end of the day belongs here". Contains standard folders for checkpoints (ckpt) and logs (log).

Storage locations

Setup

This project uses DataLad and git annex to manage stored files across various systems.

Initialization

After cloning, initialize git annex with

$ git annex init DESC

where DESC denotes a meaningful descriptor for the local repository. The data is typically stored on a clone of this repository with descriptor 'jsc' on the JSC filesystem.

Initialize the git submodules

$ git submodule update --init --recursive

You may then setup the virtual environment under env/ following the README

Adding remotes

You can add any remote clone of your new project and access the storage later with

$ git remote add REMOTE [path or ssh to remote]

Note that REMOTE has to fit the DESC name for the remote repository (e.g. jsc for the JSC filesystem) to be able to download files.

Sync the repositories

To sync with the with all added remotes and the connected GitLab project use

$ git annex sync

Get and receive files

Receive and push any file FILE from and to any source REMOTE with

$ git annex get FILE -f REMOTE
$ git annex copy FILE --to=REMOTE

List available remotes with

$ git annex whereis FILE