pyocd/cmsis-pack-manager

cmsis-pack-manager

is a python module, Rust crate and command line utility for managing current device information that is stored in many CMSIS PACKs

. Users of may query for information such as processor type, flash algorithm and memory layout information in a python program or through the command line utility, pack-manager, provided as part of this module.

CI Status

Wheels

The last step of CI uploads binary wheels to this S3 bucket.

DOCS!

They live here: https://pyocd.github.io//

Building

To build locally, Install a stable rust compiler. See https://rustup.rs/ for details on installing rustup, the rust toolchain updater. Afterwards, run rustup update stable to get the most recent stable rust toolchain and build system.

After installing the rust toolchain and downloading a stable compiler, run python2 setup.py bdist_wheel from the root of this repo to generate a binary wheel (.whl file) in the same way as we release.

For testing purposes, there is a CLI written in Rust within the rust workspace as the package cmsis-cli. For example From the rust directory, cargo run -p cmsis-cli -- update builds this testing CLI and runs the update command, for example.

Issues

Collection of the latest Issues

dvzrv

dvzrv

Comment Icon0

The setup.cfg denotes milksnake as a runtime dependency, but I believe that is very likely an error (it's used during setup, but not during runtime).

It would be great to get confirmation on this, then I can open a PR to remove it :)

dvzrv

dvzrv

Comment Icon3

Hi! It looks like maturin is the more actively maintained project, whereas milksnake appears rather dead (no release in four years, open pull requests are ignored).

Would it be possible for this project to facilitate maturin instead of milksnake (unless I'm misinterpreting the scope).

I'm asking this because I'm packaging maturin on Arch Linux already and packaging a rather undead package for building cmsis-pack-manager could potentially be circumvented.

claymation

claymation

Comment Icon0

Cache is unable to download packs in enterprise environments with "transparent" TLS-terminating middleboxes:

reqwest supports bundling Mozilla's root certificates (rusttls-tls, i.e., rust-tls-webpki-roots) or using the platform's native certificate store (rustls-tls-native-roots). See rustls-native-certs for a comparison of the two approaches.

Consider switching from rust-tls-webpki-roots to rustls-tls-native-roots to support enterprise environments with "transparent" TLS-terminating middleboxes.

mstarecek

mstarecek

bug
Comment Icon5

Hello, In version 0.3.0 I'm getting the following error when starting 'pack-manager' (or more generally 'pyocd')

OSError: cannot load library '...venv_39_32\lib\site-packages\cmsis_pack_manager_native__lib.cp39-win32.pyd': error 0xc1

This error occurs on Python 3.6 up to 3.9, but only in the 32bit versions. It would seem that the 32bit versions of wheels contain 64bit libraries because on 64bit Python everything works just fine (I've tried Python 3.6 up to 3.9)

Best Regards, Michal

flit

flit

Comment Icon0

Currently the only the entire set of PDSCs can be cached as a whole.

It would be very useful to have the abilities to separately:

  • Download only the index.
  • Determine if and which PDSCs and (downloaded) packs are out of date (or entirely new)
  • Download/update specified PDSCs.

The primary use case is to quickly check for new PDSCs and versions of already downloaded packs, and then download only those that are needed. Specifically, it would be highly useful if pyocd could quickly check for a new PDSC/pack, and potentially download it, when it is connecting to a target.

flit

flit

Comment Icon0

The Python API must return some indication of whether the requested operation succeeded, partially succeeded, or failed. Individual APIs may handle status differently.

The most difficult API is cache_descriptors(), as this very frequently encounters errors attempting to download one or more PDSCs. Similarly, it is expected to always encounter parse issues (resulting in warnings logs from the Rust code). Perhaps this is best handled with callbacks for different results.

The download_pack_list() API is easier, but can still result in partial success. Thus, a list of successfully downloaded packs should be returned.

This is required to resolve pyocd/pyOCD#1130.

flit

flit

bug
Comment Icon0

Excluding status codes handled automatically by the reqwest crate, HTTP status code are not checked by CPM. Most importantly, this includes 4xx error responses like 404 Not Found. Whatever data the server sends in the response body is still saved to disk.

The location where the response status needs to be checked is here: https://github.com/pyocd/cmsis-pack-manager/blob/7afc42405da2da525c756b1b5f23be478575ca9c/rust/cmsis-pack/src/update/download.rs#L156

flit

flit

Comment Icon3

Support integration with pack repositories from other tools:

  • Keil µVision
  • Arm CMSIS-Pack Eclipse plugin
  • IAR
  • Eclipse Embedded CDT?

Specifically, enable the features requested in this issue: ARM-software/CMSIS_5#1089.

There may be issues that prevent easy integration, but they should be identified.

flit

flit

Comment Icon0

In certain cases, the .pdsc file that is downloaded has turned out to not be XML in the PDSC schema. CPM should validate that downloaded .pdsc files are indeed XML. It doesn't need to run a full verification against the schema; simply checking for <?xml version="1.0" encoding="UTF-8"?> and the opening <package> element are sufficient.

CPM should also verify that downloaded .pack files are indeed Zip archives. It can probably use a very simple test of the magic number (see the Wikipedia article). While not completely reliable, it should hopefully be sufficient for CPM's needs due to the pack format being constrained (not arbitrary zip files).

flit

flit

Comment Icon2

(Reported by @rapgenic in #121.)

In the cmsis-pack-manager local PDSC repository, a few .pdsc files are definitely not PDSC XML. Instead they contain strange html/js code. Is this a result of a missed redirection/missing resource, or is it simply invalid configuration on the part of the silicon vendors?

List of questionable files:

  • ABOV.CM0_DFP.1.0.4.pdsc
  • ABOV.CM3_DFP.1.2.5.pdsc
  • MindMotion.MM32x031_DFP.1.0.0.pdsc
  • MindMotion.MM32x103_DFP.1.1.0.pdsc
  • Redpine.RS13100_DFP.1.0.0.pdsc
  • Redpine.RS14100_DFP.1.0.6.pdsc
  • SiliconLabs.ARTIKBG1_DFP.5.7.0.pdsc
  • SiliconLabs.ARTIKMG1_DFP.5.7.0.pdsc
  • Sinowealth.SH32Fxxx_DFP.1.0.0.pdsc
Yatekii

Yatekii

Comment Icon1

Hi!

Is there a reason that cmsis_update::install() for example requires some type with a trait implemented to be passed into it just to configure it and get some progress update? Why can't this be fixed? Is it because of the python interface? Even then I think this can be implemented with a less complicated interface which I'd highly prefer.

Also, is there a reason you force users to use slog or did you just not care enough? Because I don't want to use slog but I now have to include the dependency anyways. And exactly for this scenario there is https://docs.rs/log/0.4.8/log/ which provides a nice and unified frontend for maaaany different backends (slogbeing one of them).

I know this crate was maybe not written with intent to be reused as a lib, so don't take my criticism too harsh, I just wanted to ask and would make those improvements in a PR if that's ok =)

Best, Yatekii

flit

flit

Comment Icon1

The nRF DFP is all screwed up…

The Keil pack list site shows the latest version as 8.24.1. The pindex section of keil.vidx has a link to the nRF DFP version 8.7.1. (There is no Nordic-hosted pidx.) But the NordicSemiconductor.nRF_DeviceFamilyPack.8.7.1.pdsc file shows the most recent version is 8.26.0.

I think I understand why the keil.vidx and PDSC differ. The link to the PDSC is owned by Keil, whereas the PDSC and pack are on Nordic's site. So instead of communicating with the Keil team, Nordic has just continued to add new versions in the same PDSC without renaming the file and pack. Of course, this breaks the CMSIS-Pack versioning scheme.

When I attempt to download the nRF DFP using CPM via pyOCD, it reports this error:

The version mismatch is undoubtedly the culprit here.

I'm not really sure what CPM should do to work around this, mostly because I'm not sure where the error is actually coming from.

cc @jkrech

flit

flit

Comment Icon3

The NordicSemiconductor.nRF_DeviceFamilyPack.8.26.0 PDSC causes CPM to spew 12 of these errors:

It seems like description elements should not cause an error if missing, even though they are technically required by the schema. At least this should be changed to a warning, or better, to a debug message. (I know it doesn't stop the PDSC from being parsed, but it might unduly worry users.)

JanneKiiskila

JanneKiiskila

Comment Icon53

If you try to update the cmsis packs in Mbed OS with python project.py -m STM32F7xx --update-packs it will start downloading the pack files.

For a few ST packs, you will get the following error note:

(Edit: removed the faulty analysis of any redirects, title updated, too).

flit

flit

Comment Icon7

CPM uses apparently non-standard native library names. On Posix systems the name is _native__lib.so, while on Windows it looks like _native__lib.cp37-win_amd64.pyd.

This is only a problem because PyInstaller expects libraries to match one of the following globs: *.dll, *.dylib, lib*.pyd, and lib*.so. So basically, libs with .so and .pyd extensions are expected to have a "lib" prefix.

I've worked around it in the PyInstaller .spec file for pyOCD, but thought it was worth mentioning.

harrrson

harrrson

Comment Icon33

I've tried to install newest pyocd version on armbian, but when trying to install it even with pip, every time installed version of pack manager is 0.1. Is it possible to install newest version of pack manager on arm-based system? sorry for newbie question, but i've tried to google it, without any results.

flit

flit

Comment Icon1

The Python API needs to support a caller-provided progress callback for the cache_descriptors() and download_pack_list() methods.

Ideally the progress callback would be provided with

  • The name of the file currently being downloaded.
  • URL of current file.
  • Percent complete for current file.
  • Maybe also a total percent complete.
bridadan

bridadan

bug
Comment Icon3

Running version 0.2.6 in Python 2.7.13 on Windows 10.

The python instance crashes for me (Windows has a pop up showing that python.exe has stopped working).

The log for me looks like this:

shlomow

shlomow

Comment Icon9

Hi!

Can you please upload the source ti pypi server in addition to the wheel files? It can be useful for some use cases. Thanks in advance!

Btw, if I am trying to install the source from the releases the tab I am getting en error when building the package. I am using win10.

flit

flit

Comment Icon1

The pack index does not appear to include Dname part numbers, only Dvariant.

For instance, the NXP MIMXRT1052_DFP pack has a Dname of "MIMXRT1052xxxxB" with several Dvariants, including "MIMXRT1052CVJ5B". Only the Dvariant names are present in the index.

flit

flit

Comment Icon1

Need an API, and optionally a command line subcommand, to get the index of installed (downloaded) packs.

The existing Cache.index property should remain the same, returning all available packs.

flit

flit

Comment Icon5

The Cache constructor nominally takes a silent parameter, but this is not actually implemented. Finer log level control would also be useful.

flit

flit

Comment Icon0

It's not uncommon that users would want to remove a selected pack.

The command line interface could first show the user which devices will be removed given the device name they passed in, then ask if they really want to delete the pack. Make sure there is a --y option to force delete.

The API should just delete the pack immediately.

Versions

Find the latest versions by id

v0.4.0 - Oct 17, 2021

v0.4.0 on PyPI

Additions

  • Support for Python 3.10.
  • python-cli: add print-cache-dir subcommand (thanks @noahp!)

Changes

  • Workflows: update cibuildwheel version to v2.1.3. This builds wheels for Python 3.10, as well as using PEP 600 naming for manylinux wheels.
  • setup.cfg: bump pyyaml version to new version 6.0; other dependency cleanup.
  • Python: change CmsisPackRef to format as pack file names.

Full Changelog: https://github.com/pyocd/cmsis-pack-manager/compare/v0.3.0...v0.4.0

v0.3.0 - Sep 06, 2021

v0.3.0 on PyPI cmsis-pack crate

Additions

  • Release wheels for many more architectures, including AArch64 Linux and macOS.

Fixes

  • Add preIncludeGlobal and preIncludeLocal file categories.
  • Remove user input request before downloading packs in pack-manager tool.

Rust

  • Merge crates so cmsis-pack contains all the externally useful functionality.
  • cmsis-pack is published on crates.io now.
  • Make Core, MemoryPermissions, and Memory types public.
  • Lots of code cleanup.
  • Many dependency updates.

Python

  • Move to PEP517 project format.
  • Remove requirements files that are no longer necessary.
  • Correct project data and add classifiers.

CI

  • Entirely new CI workflows using GitHub Actions and cibuildwheel.

v0.2.10 - Mar 30, 2020

v0.2.9 - Nov 06, 2019

v0.2.8 - May 31, 2019

Improvements

  • Allow C-c during pack-downloading using the same method as pdsc files
  • Remove futures-async-await dependency
  • Compile with stable Rust >= 1.32

v0.2.7 - May 14, 2019

Improvements

  • Correct pack downloading when the "/" suffix is ommitted from the URL
  • Elide JSON dump when installing packs
  • used index.pidx in preference to Keil.pidx and Keil.vidx

v0.2.6 - Apr 10, 2019

Improvements

  • Log Errors when downloading vidx tree

v0.2.5 - Apr 08, 2019

Improvements

  • Allow C-c to cancel updates
  • Rework CFFI to force Python to handle printing
  • Add API to download a list of packs

v0.2.4 - Mar 15, 2019

Improvements

  • Continue with operations when cache is empty
  • Parse Family and Subfamily into Devices
  • Parse "0" and "1" as false and true for algorithm default attribute

v0.2.3 - Feb 01, 2019

Improvements

  • Add parsing of DVendor attribute for when the pack vendor is not the same as the device vendor

v0.2.2 - Nov 07, 2018

Fixes

  • Don't unwrap in PDSC parsing
  • Upgrade app-dirs

v0.2.1 - Aug 31, 2018

This time with Manylinux1 linux wheels

v0.2.0 - Jul 31, 2018

v0.1.0 - Dec 19, 2017

Initial public release of cmsis-pack-manager on pip.

Information - Updated Feb 11, 2022

Stars: 15
Forks: 22
Issues: 31

Robust and Fast tokenizations alignment library for Rust and Python

Blog post: maturin to build the wheel

Robust and Fast tokenizations alignment library for Rust and Python

Rust Python Integration Made Easy

PIME is a Rust crate, which allows PyO3, neotasker Python module

Rust Python Integration Made Easy

working_with_pyO3

A simple Google Kickstart question solved using Python-wrapped Rust

working_with_pyO3

A Rust clone of Python's click, built on the structopt crate

A Rust clone of Python's structopt crate

A Rust clone of Python's click, built on the structopt crate

A simple safe Rust library to cache functions (or sections of code) in memory, mainly...

A simple safe Rust library to cache functions (or sections of code) in memory, mainly ported from my Python module Rust

A simple safe Rust library to cache functions (or sections of code) in memory, mainly...

Orbit System is a Command &amp; Control system for Windows written both in the Rust...

Orbit System is a Command &amp; Control system for Windows written both in the Rust and Python programming languages

Orbit System is a Command &amp; Control system for Windows written both in the Rust...

Quick demo of getting python and rust to communicate with graphql

The main point of this is to show how we can validate a graphql query

Quick demo of getting python and rust to communicate with graphql

A simple programming language inspired by Rust and Python

The most important type in Kuru is a Set

A simple programming language inspired by Rust and Python

Python/Rust module for mol2 format (de)serialization

Default level of compression here is 3

Python/Rust module for mol2 format (de)serialization

PYO3 Python Classes in Rust

Calling Rust from Python using PyO3

PYO3 Python Classes in Rust
Facebook Instagram Twitter GitHub Dribbble
Privacy