Purpose

This page summarizes the working policy for dataquieR and QualityIndicatorFunctions development. It can be used as a reusable prompt for development sessions and as a checklist for merge requests.

Universal development prompt

You are working on dataquieR / QualityIndicatorFunctions.

Projects and issue flow:
- Public/external project: https://gitlab.com/libreumg/dataquier
- Internal development project:
  https://gitlab.com/libreumg/internal/QualityIndicatorFunctions
- User-facing issues are usually tracked in the public dataquieR project.
- Implementation usually happens in the internal QualityIndicatorFunctions
  repository.
- Internal branches and merge requests must refer back to the public issue.

Branch and merge request convention:
- Use branches named issueXXX_short_description, for example
  issue803_report_without_metadata.
- Open a draft merge request in the internal QualityIndicatorFunctions project.
- The MR description should link the public issue, for example:
  Fixes libreumg/dataquieR#803
  or, if the work is not final:
  Refs libreumg/dataquieR#803
- Do not do the final merge automatically; the final merge is reviewed together
  with Stephan.
- Small, clear CI fixes on the same branch are acceptable. Re-run focused local
  checks before pushing.

Code conventions:
- Read the surrounding code first and follow existing local patterns.
- Keep changes narrowly scoped to the issue.
- Internal helper functions use the util_ prefix.
- Do not introduce abstractions unless they remove real complexity or match an
  established package pattern.
- Use the exported metadata name constants whenever they exist. Metadata
  column and attribute names such as `DF_CODE`, `VAR_NAMES`, `DATA_TYPE`, or
  names from `WELL_KNOWN_META_VARIABLE_NAMES` must not be duplicated as string
  literals in implementation code. For example, prefer
  `meta_data[[VAR_NAMES]]` over `meta_data[["VAR_NAMES"]]`, and prefer
  `meta_data[[VAR_NAMES]]` over `meta_data$VAR_NAMES`.
- Exported functions must be robust and user-facing errors must be meaningful.
  Avoid raw R errors where a controlled dataquieR condition or explanatory
  message is possible.
- Unexpected failures should be reported in a way that helps users contact the
  maintainers with enough context.
- Do not edit the external dataquieR repository directly when the internal
  QualityIndicatorFunctions repository is the upstream source.
- Do not change versions, NEWS, release notes, tags, or releases unless release
  work was explicitly requested.

Roxygen and generated artifacts:
- If Roxygen documentation changes, run devtools::document().
- Roxygen updates can regenerate inst/manual.RData and
  inst/indicator_or_descriptor.RData.
- These binary artifacts may be committed to the branch that regenerated them.
- After merges that touch Rd files, document() may need to be run again on the
  target branch, because these generated artifacts can change.

CRAN policy and tests:
- CRAN-active tests must not depend on unstable network access, external
  services, fragile snapshots, or long-running system state.
- Use skip_on_cran() or targeted skip logic for tests that are slow, online-only,
  environment-specific, vdiffr-based, or otherwise fragile.
- CRAN restricts parallel execution; do not require more than two cores in tests.
- If we want to detect unstable upstream changes early, prefer CI/MR pipelines
  over CRAN-active tests.
- Avoid adding CRAN-active tests that can fail because of dependency internals,
  platform rendering, or hash details outside our control.

Local checks before pushing:
- Run the smallest meaningful focused test first, for example:
  devtools::test(filter = "some_context", stop_on_failure = TRUE)
- If documentation changed, run devtools::document().
- If user-facing text, NEWS, or documentation changed, run
  devtools::spell_check().
- For broader changes, consider:
  devtools::check(args = "--no-tests")
- Keep generated reports, logs, and example artifacts in an obvious location if
  Stephan should review them locally.

CI/CD handling:
- Watch GitLab pipelines after pushing branches or starting releases.
- Retry transient runner, network, or timeout failures if appropriate.
- If Nexus is full and access is available, clean up cautiously; otherwise inform
  Stephan.
- Fix small, clear spell-check, Roxygen, or focused test failures directly on the
  branch.
- If the failure is unclear or risky, stop and inform Stephan.
- Do not delete or stop pipelines, tags, releases, or artifacts unless this is
  clearly part of the requested cleanup.
- For vdiffr snapshot work, use the internal
  libreumg/internal/vdiffr-snapshot-manager helper where possible. It can
  download GitLab job artifacts, collect render-stack metadata, run local
  Docker-based checks, start vdiffr::snapshot_review(), and run temporary
  remote snapshot pipelines via `bin/test-current-snapshot-remotely.sh`.
- CI container snapshots are authoritative. Snapshots generated on developer
  machines are useful diagnostics, but they must not be accepted only because a
  local run produced `.new.svg` files. Accept snapshot changes only when the
  corresponding GitLab CI container/image for the relevant `R_VERSION` produces
  the same change, or when a deliberate rendering-stack migration is being
  reviewed.
- For maintaining R package CI/CD projects based on ci-scripts and ci-templates,
  use the manageGitlabRProjects package from ~/git/gitlab or Nexus.

Release and beta policy:
- Start alpha, beta, or release pipelines only when explicitly requested.
- Follow R versioning conventions, e.g. 2.8.10.9001 for a beta/development
  version.
- Use RELEASE_TAG=vX.Y.Z... deliberately.
- `RELEASE_TAG` may include the human-readable release title after the version,
  e.g. `v2.8.10.9001 New Report Rendering and faster computation`. The first
  whitespace-separated token is the version tag; the full value can still be
  used as commit message and release title.
- Before rerunning a release/beta pipeline for the same version, check and clean
  duplicate tags, releases, and Nexus artifacts where needed.
- Follow downstream pipelines started by external dataquieR, including child
  pipelines, dataquieRlauncher, rstudio/docker/rstudio, DockerHub and
  Nexus-related builds.
- On GitLab Free/Community Edition, release authorization should not rely on
  Protected Environments. Use protected default branches, protected release
  secrets, a manual release gate, and the separate
  [qif-release-policy](https://gitlab.com/libreumg/internal/qif-release-policy)
  repository instead.
- The release pipeline must check all of the following before syncing to the
  external dataquieR repository:
  - it runs on the default branch;
  - `CI_COMMIT_REF_PROTECTED` is `true`;
  - protected release secrets such as `RELEASE_TARGET` and `RELEASE_SSH_FILE`
    are present;
  - `GITLAB_USER_LOGIN` is listed in
    `qif-release-policy:qif-release-allowed-users.txt`;
  - the current QIF `.gitlab-ci.yml` SHA-256 hash matches
    `qif-release-policy:qif-gitlab-ci.sha256`;
  - the version encoded in `RELEASE_TAG` matches `DESCRIPTION`;
  - the release version is either `Major.Minor.Patch` or `Major.Minor.Patch.900N`
    and not an alpha version such as `Major.Minor.Patch.900N.K`.
- The policy repository contains:
  - `qif-release-allowed-users.txt`: GitLab usernames allowed to start QIF
    releases, one per line;
  - `qif-gitlab-ci.sha256`: reviewed hash of the QIF `.gitlab-ci.yml`;
  - `scripts/update-qif-ci-reference.sh`: helper to update the hash after an
    intentional QIF CI change has been reviewed.
- The QIF job token must be allowed to read the policy repository. Configure the
  policy project allowlist with numeric project IDs:
  ```sh
  QIF_PROJECT_ID=10015470
  POLICY_PROJECT_ID=84252717

  glab api \
    --method POST \
    "projects/${POLICY_PROJECT_ID}/job_token_scope/allowlist" \
    --field target_project_id="${QIF_PROJECT_ID}"

  glab api "projects/${POLICY_PROJECT_ID}/job_token_scope/allowlist" \
    | jq -r '.[] | [.id,.path_with_namespace] | @tsv'

  • To approve an intentional QIF .gitlab-ci.yml change:

    cd /Users/struckmanns/git/gitlab/qif-release-policy
scripts/update-qif-ci-reference.sh /Users/struckmanns/git/gitlab/QualityIndicatorFunctions
git diff
git add qif-gitlab-ci.sha256
git commit -m "Update QIF CI release policy reference"
git push

  • Be honest about the Free/Community Edition limitation: a QIF maintainer can still remove the policy check from QIF .gitlab-ci.yml; the external policy repository is a guardrail against accidental or casual changes, not a cryptographic root of trust. Put the policy repository in a group/subgroup with a deliberately small maintainer set. For stronger selected-subgroup authorization, use GitLab Premium/Ultimate Protected Environments.

  • QIF-specific operational note: as of the July 2026 audit, master is protected for maintainer-only push and merge. RELEASE_SSH_FILE is protected, but RELEASE_TARGET was not protected and should be made protected before relying on the release gate. The initial qif-release-policy project was created under libreumg/internal; because group maintainers inherit project maintainer rights, it should be moved to a narrower group/subgroup if the inherited maintainer set remains too broad.

dataquieR-specific behavior: - Grading rulesets should be applied as late as possible, preferably during rendering/plotting, so already computed result objects can still be displayed differently when rulesets change. - If several grading rules classify the same contradiction rule, the worst class, i.e. the highest class, wins. - Keep item-level and cross-item-level logic separate. - Intrinsic applicability problems are expected non-applicability, not ordinary crashes. They should render neutrally but keep their explanatory condition message. - Missing or not-requested assessments should be understandable in reports. - Robustness of exported functions is more important than performance shortcuts. Performance improvements must not reduce controlled behavior or explanatory messages.

Working style: - Use small, reviewable commits. - Do not revert unrelated work in a dirty worktree. - Commit only files that belong to the current task. - Prefer focused tests and concrete local reproductions. - Keep status updates short and factual: what was checked, what changed, and what remains. ```

Checklist for internal merge requests

  • Branch follows issueXXX_short_description.
  • Draft MR exists in the internal project.
  • MR links the public dataquieR issue.
  • Local focused tests are listed in the MR.
  • devtools::spell_check() was run if user-facing text changed.
  • devtools::document() was run if Roxygen changed.
  • CRAN-active tests are stable and do not require network access, fragile snapshots, or more than two cores.
  • Final merge is left for joint review.

Developer helper tools

The following helper tools support the workflow above:

  • vdiffr-snapshot-manager: internal helper for reviewing and accepting vdiffr snapshots. It can download GitLab CI artifacts, copy .new.svg files into a local checkout, start vdiffr::snapshot_review(), run snapshot tests inside Docker CI images, write render-stack metadata, and run bin/test-current-snapshot-remotely.sh to start a temporary remote pipeline for the current snapshot state. That shell workflow downloads all relevant artifacts locally and deletes the temporary pipeline afterwards. Use it when the question is whether the remote CI image produces the same snapshot output as a local run. The CI image is the source of truth for accepting snapshots; local developer-machine output is only supporting evidence. Snapshot acceptance and pushing are explicit actions.
  • manageGitlabRProjects: R package for creating and maintaining R packages that use the shared ci-scripts and ci-templates GitLab CI/CD setup. The package is available in ~/git/gitlab and in Nexus.

For snapshot provenance, CI jobs should store vdiffr-render-metadata.json as an artifact and print the compact render-stack summary to the job log. This file should include at least R, platform, relevant graphics packages, CI image/tag, and discoverable system graphics libraries.