A Google Group has been created to facilitate discussion on TARDIS, and where it's headed in the future.
TARDIS | Google Groups
Sunday, October 12, 2008
Tuesday, September 16, 2008
Wiki Created
A wiki has been created to hold information on future development, foster collaboration and feedback from the crystallography community, metadata experts, repository developers/administrators and anyone else who is interested in our progress.
Wiki
Wiki
Monday, September 1, 2008
DatasetTools Update
A minor update has been made to DatasetTools, with a fix to Oscillation Range extraction, and a slight change of schema.
Please update your version by re-downloading the software on the Tools page.
Please update your version by re-downloading the software on the Tools page.
Thursday, August 21, 2008
TARDIS Portal Out Of Beta
Currently harvesting datasets out of the Monash Digital Repository is the TARDIS Portal, which was just brought out of beta.
To see the current indexed datasets, see: Data Portal.
To see the current indexed datasets, see: Data Portal.
Sunday, June 22, 2008
TARDIS Beta Portal
Work on the very first version of the TARDIS portal is in beta. It currently scans the Monash Test Repository for crystallography datasets and harvests their information for searching/downloading from this site.
Data Portal Beta
Data Portal Beta
Tuesday, June 17, 2008
Acta Cryst. Paper Online
Our Acta Cryst paper, "Federated repositories of X-ray diffraction images" is now online!
http://www.ncbi.nlm.nih.gov/pubmed/18566516?ordinalpos=1&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_RVDocSum
http://www.ncbi.nlm.nih.gov/pubmed/18566516?ordinalpos=1&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_RVDocSum
Sunday, May 4, 2008
Dataset Tools Released
The Dataset Tools suite has been released.
Download
Additionally, a tutorial video has been created demonstrating the tools:
Tutorial Video
Download
Additionally, a tutorial video has been created demonstrating the tools:
Tutorial Video
Wednesday, April 16, 2008
TARDIS Mentioned In Acta Cryst
The TARDIS web site has been mentioned in the Acta Crystallographica Section D editorial this month.
Find it here: http://scripts.iucr.org/cgi-bin/paper?S0907444908004915
Find it here: http://scripts.iucr.org/cgi-bin/paper?S0907444908004915
Thursday, April 10, 2008
Site Revamp
The site's gone through some drastic cosmetic changes in the last couple of days in preparation for the release of Dataset Tools. The tools are considered 'done' and will be released very soon. For now, there are screenshots and descriptions on the Tools page.
There's also an updated metadata schema for datasets, coupled with an example produced using Dataset Tools.
There's also an updated metadata schema for datasets, coupled with an example produced using Dataset Tools.
Thursday, February 21, 2008
Deposition Tools Coming Soon!
Update: Dataset Tools has been released
A set of desktop tools that will make depositing crystallography datasets into a Fedora 2.2.1 repository easier are about to be released in beta on this site. Fedora is the fundamental persistent storage technology behind digital repository software such as Arrow and Fez.
Each tool was programmed in Java for cross-platform compatibility, each having a GUI version as well as command-line functionality. Together they aim to make it easier for researchers to package their data, create valid metadata, both technical and repository-based, upload the data and then unpackage it again once downloaded. The GUI tools are:
RepositoryPackager
Created to take a set of diffraction images, tar archive them, bzip2 compress them and then split the resultant file on a chosen file size. This is done in an effort to make data more space efficient in a repository, and because large files (over 2GB) were found to crash various server software in the upload/storage process.
Uses the apache tar/bzip2 java libraries and incorporates the accompanying command-line tools TarBzipper and FileSplitter.
METSManager
Fedora repositories ingest METS XML files to create and describe entries being stored. This program uses the Harvard METS Java Toolkit to create a fedora-compatible METS package that includes entered values for ingestion. Currently, DublinCore data is created within the METS package to describe basic values such as the title of the object in the repository and the authors. Technical xml metadata relating to the experiment itself is currently put as plain text in the Description field.
Created METS XML files can be validated against the Fedora-compatible METS schema to test their eligibility for ingestion.
Future plans include the embedding of technical XML data such as that described in the data section of this site for data harvesting.
Is a GUI implementation of the accompanying command-line tool METSCreator.
DataDepositor
Once data is organised for upload, and a METS XML file is created, the object and its data are ready to be ingested and uploaded into the repository. This program makes such a process simple by scanning a supplied directory (non-recursive since the structure of repository objects are flat) and uploading each file into a new object described by the METS XML file.
Uses the fedora-management API, and is a GUI implementation of the accompanying command-line tool of the same name.
Currently only compatible with Fedora 2.2.1 repositories.
RepositoryUnPackager
Once a split set of files created by RepositoryPackager are downloaded again from the repository, they can be re-joined, uncompressed and unarchived to restore them to their original form using this program.
Is a GUI implementation of the accompanying command-line tools FileJoiner and UnTarBzipper.
-
These tools will be open-source and hosted on SourceForge for the community to freely use, view and modify. Additionally a user guide will be created to guide users through the process of using the tools, and also for setting up a compatible Fedora repository.
Expect this site to be updated with beta versions of the tools within the next 2 weeks.
A set of desktop tools that will make depositing crystallography datasets into a Fedora 2.2.1 repository easier are about to be released in beta on this site. Fedora is the fundamental persistent storage technology behind digital repository software such as Arrow and Fez.
Each tool was programmed in Java for cross-platform compatibility, each having a GUI version as well as command-line functionality. Together they aim to make it easier for researchers to package their data, create valid metadata, both technical and repository-based, upload the data and then unpackage it again once downloaded. The GUI tools are:
RepositoryPackager
Created to take a set of diffraction images, tar archive them, bzip2 compress them and then split the resultant file on a chosen file size. This is done in an effort to make data more space efficient in a repository, and because large files (over 2GB) were found to crash various server software in the upload/storage process.
Uses the apache tar/bzip2 java libraries and incorporates the accompanying command-line tools TarBzipper and FileSplitter.
METSManager
Fedora repositories ingest METS XML files to create and describe entries being stored. This program uses the Harvard METS Java Toolkit to create a fedora-compatible METS package that includes entered values for ingestion. Currently, DublinCore data is created within the METS package to describe basic values such as the title of the object in the repository and the authors. Technical xml metadata relating to the experiment itself is currently put as plain text in the Description field.
Created METS XML files can be validated against the Fedora-compatible METS schema to test their eligibility for ingestion.
Future plans include the embedding of technical XML data such as that described in the data section of this site for data harvesting.
Is a GUI implementation of the accompanying command-line tool METSCreator.
DataDepositor
Once data is organised for upload, and a METS XML file is created, the object and its data are ready to be ingested and uploaded into the repository. This program makes such a process simple by scanning a supplied directory (non-recursive since the structure of repository objects are flat) and uploading each file into a new object described by the METS XML file.
Uses the fedora-management API, and is a GUI implementation of the accompanying command-line tool of the same name.
Currently only compatible with Fedora 2.2.1 repositories.
RepositoryUnPackager
Once a split set of files created by RepositoryPackager are downloaded again from the repository, they can be re-joined, uncompressed and unarchived to restore them to their original form using this program.
Is a GUI implementation of the accompanying command-line tools FileJoiner and UnTarBzipper.
-
These tools will be open-source and hosted on SourceForge for the community to freely use, view and modify. Additionally a user guide will be created to guide users through the process of using the tools, and also for setting up a compatible Fedora repository.
Expect this site to be updated with beta versions of the tools within the next 2 weeks.
Sunday, February 3, 2008
Site Update
Some notable bits and pieces have been added to this site, reflecting progress towards the goal of online-hosted datasets that this site will index.
Three More Experiments Hosted
Monash's ARROW library repository site is now hosting three more experiments containing complete datasets. They are linked here on the data page.
Dataset XML Schemas Updated, Example Added
The metadata schemas for diffraction image datasets have been updated - mostly to conform fully to the XML-schema for XML schemas defined by the W3. Additionally an example of experimental/instrument-related metadata descriptions that conform to this schema has been included as a guideline.
Other News
Tools are currently being developed to extract instrument data from image files into TARDIS-compatible XML metadata. Tools are also being developed to compress and split large collections of images in order to be more suitable by internet repositories.
This blog will be refreshed often with development updates.
Three More Experiments Hosted
Monash's ARROW library repository site is now hosting three more experiments containing complete datasets. They are linked here on the data page.
Dataset XML Schemas Updated, Example Added
The metadata schemas for diffraction image datasets have been updated - mostly to conform fully to the XML-schema for XML schemas defined by the W3. Additionally an example of experimental/instrument-related metadata descriptions that conform to this schema has been included as a guideline.
Other News
Tools are currently being developed to extract instrument data from image files into TARDIS-compatible XML metadata. Tools are also being developed to compress and split large collections of images in order to be more suitable by internet repositories.
This blog will be refreshed often with development updates.
Subscribe to:
Comments (Atom)