thoth-pub/thoth

Open bibliographic metadata management and dissemination system

GraphQL API, implementing a data model specifically designed for OA books

Thoth

About

Thoth (/toʊt, θoʊθ/, Greek Θώθ < Coptic Ⲑⲱⲟⲩⲧ < Egyptian ḏḥwtj) is an Open Dissemination System for Open Access books. Written purely in rust, it consists of:

  • A GraphQL API, implementing a data model specifically designed for OA books
  • A REST API to export metadata in formats like ONIX, MARC, etc.
  • A WebAssembly GUI to manage metadata records.

For more information about Thoth, its data and metadata formats, and more, see the repo's wiki. You can also use GraphiQL to explore the GraphQL API (click on "Docs" at the top right), or RapiDoc to inspect the REST API.

Getting Started

Requirements

  • Rustup
  • Stable Toolchain: rustup default stable
  • wasm-pack
  • rollup
  • A PostgreSQL database (included in docker-compose.yml if ran using docker)
  • libssl-dev

Running with docker

git clone https://github.com/thoth-pub/thoth.git
cd thoth
cp .env.example .env  # Edit the credentials in .env
docker-compose up

Running with rust (cargo)

Config

git clone https://github.com/thoth-pub/thoth.git
cd thoth
cp .env.example .env  # Edit the credentials in .env

Creating Postgres DB and User

psql
psql -U postgres
CREATE ROLE thoth SUPERUSER LOGIN PASSWORD 'thoth';
CREATE DATABASE thoth WITH OWNER thoth; 

Exit the psql command line with:

\q

An example of a .env file:

THOTH_GRAPHQL_API=http://localhost:8000
# THOTH_EXPORT_API is used at compile time, must be a public facing URL
THOTH_EXPORT_API=http://localhost:8181
# Authentication cookie domain
THOTH_DOMAIN=localhost
# Full postgres URL (With the role and db we created in the orevious step, it will look like this)
DATABASE_URL=postgres://thoth:[email protected]/thoth
# Authentication cookie secret key (can be any string really)
SECRET_KEY=we_like_s%_books_255
# Logging level
RUST_LOG=info

Proceed to install Wasm

cargo install wasm-pack

Install rollup:

sudo npm install --global rollup

Export the ENV variables, THOTH_GRAPHQL_API and THOTH_EXPORT_API:

export THOTH_GRAPHQL_API=http://localhost:8000
export THOTH_EXPORT_API=http://localhost:8181

API

cargo run init

Wasm GUI

wasm-pack build thoth-app/ --target web \
  && rollup thoth-app/main.js --format iife --file thoth-app/pkg/thoth_app.js \
  && cargo run start app

Building with docker

The wasm APP needs to know the endpoint the API will be running at compile time, we must provide THOTH_API as a build argument to the docker daemon upon build:

docker build \
    --build-arg THOTH_GRAPHQL_API=https://api.thoth.pub \
    --build-arg THOTH_EXPORT_API=https://export.thoth.pub \
    . -t openbookpublishers/thoth

Acknowledgements

Thoth is being developed as part of the COPIM project, an international effort to build community-owned, open systems and infrastructures to enable Open Access book publishing to flourish. COPIM is funded by the Research England Development (RED) Fund, and Arcadia, a charitable fund of Lisbet Rausing and Peter Baldwin.

Issues

Collection of the latest Issues

ja573

ja573

enhancement
Comment Icon0

At present the page_interval field is populated using format!("{}-{}", first, last). We should replace the hyphen with an en-dash

rhigman

rhigman

feature
Comment Icon0

User feedback: it's time-consuming to add the same basic metadata (copyright, author, institution, etc) for multiple chapters of a parent work. Potential solution: add "create child" button which duplicates the parent record - the new record can then be minimally edited as necessary (copy cataloguing).

rhigman

rhigman

bug
Comment Icon1

It is possible to enter a value https://creativecommons.org/ licenses/by-nc/4.0/ in the work Licence field (note space between .org/ and licences), even though both the app and the database have constraints which should prevent invalid URL values being entered.

This is not caused by #376 as the behaviour is the same when both adding and editing works. Similar issues exist for the Landing Page, Cover URL, and publication location Landing Page and Full Text URL fields. The bug is presumably in our generic implementation of URL constraints.

lb803

lb803

bug
Comment Icon7

Describe the bug The Crossref Metadata Input Schema v. 5.3.1 accepts max 6 ISBNs in a single book DOI deposit. At present, Thoth doesn't seem to limit the number of ISBNs in the CrossRef metadata output.

To Reproduce Steps to reproduce the behavior:

  1. $curl -X GET "https://export.thoth.pub/specifications/doideposit%3A%3Acrossref/work/2566ed3d-3df1-4e56-a7dc-455ecf7d3a4b"
  2. The output reports 7 ISBNs associated to the work.

Expected behavior Limit the number of ISBNs to 6 in output.

Desktop (please complete the following information):

  • OS: Linux
  • Curl
  • Version 7.68.0

Additional context Thank you!

rhigman

rhigman

bug
Comment Icon1

This is another bug caused by https://github.com/jgthms/bulma/issues/3390, related to #228/#229, introduced under #325.

Each item in a Work page's Related Works list has a View button allowing the user to navigate directly to the Work (similarly to the Publications list). Because the Work component is already loaded for the current Work, the page is not refreshed/recreated, but the details of the newly-selected Work are instead retrieved from the database and loaded into the existing component. This is currently the only instance where a user can navigate directly from one instance of an object to another instance of the same object.

Because Works have some dropdown values (e.g. Work Type, Work Status, Imprint), this leads to us potentially changing the underlying <selected> values of these dropdowns via code rather than through the interface. This means we may hit the Bulma bug whereby the displayed dropdown selection can go out of sync with the underlying value.

As an example, in testing, I navigated from a monograph with test imprint "C" to a related edited book with test imprint "H". Whereas inspecting the dropdowns with browser developer tools showed that they were correctly set to "edited book" and "H", their displayed values were "Book Chapter" and "Select Imprint" (the topmost values in the list).

(This did not occur until after a few rounds of clicking between related works, and is not consistently repeatable. The status for both happened to be "Inactive", and this was displayed correctly throughout. Also of interest, I tried clicking "Save" at this point, and was given the GUI warning "Please select an imprint", although an imprint was underlyingly selected.)

It is not clear why this issue does not appear to arise when a component is initially created, as this process uses similar logic of creating a default Work object and then updating it (including updating its dropdown selections) as soon as the correct database values are retrieved.

rhigman

rhigman

feature
Comment Icon2

WP7 have secured access to the Bath Spa university repository, which implements EPrints, and permits API interaction using the SWORD (v2) protocol.

Similarly to #289, investigate/report on possibilities of using this for automated upload/archiving.

ja573

ja573

feature
Comment Icon5

We would like to add information about the editor(s) of a series.

There are two options:

  • we reuse the existing contribution + affiliation schema, adding the series_id to contribution and constraining the types (e.g. can only add EDITOR as the type if there is a series_id, there can only be either a work_id or a series_id but not both)
  • we recreate the structure, adding a series_editor object with similar fields to contribution, and a series_editor_affiliation with similar fields to affiliation
ja573

ja573

external relations
Comment Icon2

We have an output for individual publisher for OCLC

To create a service for a group of publishers we need: to be able to output data for multiple publishers have a process for error detection and correction

ja573

ja573

feature
Comment Icon0

Investigate using Kafka instead of creating it from scratch. Investigate if Kafka connect could replace event composition in the GraphQL API

rupertgatti

rupertgatti

feature
Comment Icon0

What is our backup strategy for Thoth? How often have we tested it? Introduce regular test framework/timetable

Versions

Find the latest versions by id

v0.8.6 - Jul 01, 2022

[0.8.6] - 2022-07-01

Added

  • #390 - Implement OverDrive ONIX 3.0 specification

Fixed

  • #392 - Fix encoding of print ISBN in JSTOR ONIX output

v0.8.5 - May 30, 2022

[0.8.5] - 2022-05-30

Added

  • #287 - Allow editing contributions (and affiliations)

Fixed

  • #360 - Prevent adding 0 as the price of a publication
  • #376 - Restrict Licence field entries to URL-formatted strings

v0.8.4 - May 11, 2022

[0.8.4] - 2022-05-11

Added

  • #29 - Implement CrossRef DOI Deposit specification
  • #72 - Implement CrossRef Google Books ONIX 3.0 specification

Changed

  • #356 - Upgrade actix to v4

v0.8.3 - Apr 18, 2022

[0.8.3] - 2022-04-18

Added

  • #359 - Allow editing publications

v0.8.2 - Apr 06, 2022

[0.8.2] - 2022-04-06

Changed

  • Added CA certificates to docker image to allow https requests from containers

v0.8.1 - Mar 11, 2022

[0.8.1] - 2022-03-11

Added

  • #104 - Implement BibTeX specification

Changed

  • Removed unnecessary title branching logic from KBART/ONIX output formats

v0.8.0 - Mar 01, 2022

[0.8.0] - 2022-03-01

Added

  • #341 - Add weight to publication

Changed

  • Tidied verbose bools and single-character strings to comply with rustc 1.59.0
  • #300 - Moved width/height to Publication, added depth, improved metric/imperial display
  • Upgrade docker's base images to latest available releases

v0.7.2 - Feb 08, 2022

[0.7.2] - 2022-02-08

Changed

  • #339 - Update publication types to include AZW3, DOCX and FictionBook
  • #331 - Update series model to include description and CFP URL
  • Allow triggering docker action manually

Added

  • Add code of conduct and support document to repository

v0.7.1 - Jan 24, 2022

[0.7.1] - 2022-01-24

Changed

  • Removed redundant to_string calls to comply with rustc 1.58.0
  • #329 - Update EBSCO Host ONIX pricing and contributor display logic
  • Allow building docker image manually in actions

v0.7.0 - Jan 11, 2022

[0.7.0] - 2022-01-11

Added

  • #28 - Implement chapter structure
  • GraphQL queries: support filtering on multiple enum variants (e.g. work types, language codes)
  • Dashboard: display Institution stats

Fixed

  • Issues form: typing filter string in series search box has no effect on which series are displayed

v0.6.1 - Dec 13, 2021

[0.6.1] - 2021-12-13

Changed

  • Removed redundant closures and impls to comply with rustc 1.57.0

Fixed

  • #309 - Update Thema codes to v1.4

v0.6.0 - Nov 29, 2021

[0.6.0] - 2021-11-29

Added

  • #92 - Implement institution table, replacing funder and standardising contributor affiliations

v0.5.0 - Nov 29, 2021

[0.5.0] - 2021-11-28

Added

  • #297 - Implement publication location

Changed

  • Requirement to Number fields preventing user from entering numbers below 0 for Counts/below 1 for Editions and Ordinals, and sets Contribution Ordinal default to 1 instead of 0
  • #299 - Update Project MUSE ONIX subject output logic
  • Updated if and else branches to comply with rustc 1.56.0

Fixed

  • #292 - Cannot unset pubiication date: error when trying to clear a previously set publication date
  • #295 - various subforms failing to trim strings before saving (including on mandatory fields which are checked for emptiness)
  • Duplicated logic for handling optional field values, simplifying the code and reducing the likelihood of further bugs such as
  • Minor issue where some required fields were not marked as "required" (so empty values would be sent to the API and raise an error)
  • Issue with subforms where clicking save button bypassed field requirements (so instead of displaying a warning message such as "Please enter a number", invalid values would be sent to the API and raise an error)
  • #310 - Add jstor specification to formats

v0.4.7 - Oct 04, 2021

[0.4.7] - 2021-10-04

Added

  • #43, #49 - Implement EBSCO Host's ONIX 2.1 specification
  • #44 - Implement JSTOR's ONIX 3.0 specification
  • #253 - Implement Project MUSE ONIX specification tests

Changed

  • #242 - Move API models to object-specific subdirectories
  • #274 - Add width/height units to CSV specification
  • #263 - Add Doi, Isbn and Orcid types to client schema

v0.4.6 - Sep 02, 2021

[0.4.6] - 2021-09-02

Added

  • #88 - Implement KBART specification
  • #266 - Delete confirmation to publications

Changed

  • #272 - Use more fields in contributors filtering

Fixed

  • #271 - Make filter parameter optional in subjectCount

v0.4.5 - Aug 12, 2021

[0.4.5] - 2021-08-12

Added

  • #259 - Units selection dropdown to Work and NewWork pages, which updates the Width/Height display on change
  • #259 - Local storage key to retain user's choice of units across all Work/NewWork pages
  • #259 - Backend function to convert to/from database units (mm): uses 1inch = 25.4mm as conversion factor, rounds mm values to nearest mm, rounds cm values to 1 decimal place, rounds inch values to 2 decimal places and then to nearest sixteenth of an inch
  • #259 - Constraints on Width/Height fields depending on unit selection: user may only enter whole numbers when in mm, numbers with up to 1 decimal place when in cm, numbers with up to 2 decimal places when in inches

Changed

  • #259 - GraphQL and APP queries to specify units when submitting new Width/Height values, and handle conversion if required

v0.4.4 - Aug 02, 2021

[0.4.4] - 2021-08-02

Fixed

  • Read button in catalogue now uses the landing page URL instead of the DOI

Changed

  • Removed needless borrow to comply with clippy under rustc 1.54.0

v0.4.3 - Jul 28, 2021

[0.4.3] - 2021-07-28

Added

  • #48 - Implement OAPEN ONIX 3.0 specification

Fixed

  • #254 - Ensure order of fields in create work match those in edit work

v0.4.2 - Jul 05, 2021

[0.4.2] - 2021-07-05

Added

  • #125 - Implement ISBN type to standardise parsing
  • #217 - Add "Contribution Ordinal" field to indicate order of contributions within a work

v0.4.1 - Jun 22, 2021

[0.4.1] - 2021-06-22

Changed

  • #234 - Move database calls out of GraphQL model

Added

  • #136, #233 - Implement Doi and Orcid types to standardise parsing
  • thoth-errors crate to share ThothError and ThothResult

v0.4.0 - Jun 15, 2021

[0.4.0] - 2021-06-14

Changed

  • Updated yew to v0.18.0
  • Updated actix-web to 3.3.2
  • Catch client errors with ThothError::EntityNotFound
  • Use a custom instance of GaphiQL

Added

  • #235 - Export API with openapi schema
  • #110 - Output to CSV
  • Rapidoc schema explorer interface

Removed

  • actix_rt

v0.3.6 - May 11, 2021

[0.3.6] - 2021-05-11

Fixed

  • Problem building docker image

v0.3.5 - May 11, 2021

[0.3.5] - 2021-05-11

Added

  • #213 - Link to documentation in readme
  • #206 - Notify user when a new version of the APP is available
  • #231 - Link to publication page in work page
  • #224 - Implement limit and offset in linked queries
  • Implement Crud trait with database calls per object

Changed

  • #236 - Split server logic into individual crates
  • Update rustc to 1.51.0 in docker image
  • Replace composite keys in contribution and issue with standard UUIDs
  • Server configuration parsed from binary

Fixed

  • #216, #228 - Error adding multiple subjects

v0.3.4 - Mar 29, 2021

[0.3.4] - 2021-03-29

Fixed

  • Upgraded rusct in docker image. Moved wasm-pack to a less fragile build stage using official image, keeping main build statically compiled

v0.3.3 - Mar 26, 2021

[0.3.3] - 2021-03-26

Added

  • #120 - Implement table sorting by columns in APP
  • #203 - Cascade filtering options to relation queries in API

Changed

  • #210 - Specify .xml extension when outputting ONIX files

Fixed

  • #182 - Ensure issue's series and work have the same imprint

v0.3.2 - Mar 09, 2021

[0.3.2] - 2021-03-09

Added

  • #202 - Enum type filtering in GraphQL queries
  • #202 - Query works by DOI
  • #195 - Prompt confirmation upon delete

Fixed

  • #199, #201 - Error displaying publications if filtering on empty ISBN or URL
  • Trigger a warning when the current user does not have any editting permissions

v0.3.1 - Mar 04, 2021

[0.3.1] - 2021-03-04

Fixed

  • #197 - Error deserialising publications in APP

v0.3.0 - Mar 03, 2021

[0.3.0] - 2021-03-03

Changed

  • #162 - Only records linked to publishers user has access to are listed in APP
  • #167 - Make work contribution the canonical source of contributor names in ONIX output

Added

  • #177 - Allow querying objects by linked publisher(s)
  • #159, #160, #161 - Add publisher accounts
  • #163 - Save a snapshot of each object upon update
  • #164, #165 - Add contributor names to contribution
  • #168 - Warn users when editing a contributor or a funder that is linked to a work
  • #185 - Allow resetting user passwords through CLI
  • Allow creating publisher accounts through CLI

Fixed

  • #181 - Enforce numeric values for issue ordinal

v0.2.13 - Jan 14, 2021

[0.2.13] - 2021-01-14

Changed

  • Update API URL in docker github action
  • Remove staging tag in docker github action

v0.2.12 - Jan 12, 2021

[0.2.12] - 2021-01-12

Changed

  • #153 - Implement created and updated dates to each structure

Information - Updated Jul 26, 2022

Stars: 25
Forks: 7
Issues: 104

Repositories & Extras

Percy lets you build frontend browser apps with Rust + WebAssembly

Supports server side rendering as-well and has a huge developer community for both bug fixing and support

Percy lets you build frontend browser apps with Rust + WebAssembly

Rust WebAssembly A* Pathfinding Demo

This is a port of an A* implementation of mine from an old Unity maze project

Rust WebAssembly A* Pathfinding Demo

yew-train-ticket

A Rust WebAssembly Websit example to buy train ticket using yew hooks

yew-train-ticket

Rust-generated WebAssembly GitHub action template

A template to bootstrap the creation of a Rust-generated WebAssembly GitHub action

Rust-generated WebAssembly GitHub action template

Rust Web assembly game 1024

The game logic has been developed by Rust Programming Language

Rust Web assembly game 1024

approx-string-match-rs

A Rust + WebAssembly library for approximate string matching

approx-string-match-rs

first-rust-webassembly

My first rust/webassembly experiment

first-rust-webassembly

Markdown editor built using Rust WebAssembly

Run wasm-pack build --target web to compile rust into wasm

Markdown editor built using Rust WebAssembly

Rust WebAssembly implementation of an old board game with a board similar to

scrabble, but instead of placing words, players build a river out of tiles with

Rust WebAssembly implementation of an old board game with a board similar to
Facebook Instagram Twitter GitHub Dribbble
Privacy