проектные команды и ссылки на источник

- заведён internal/source: уровни ссылаются друг на друга путём на диске
  или git-репозиторием, ревизия закрепляется хвостом #ref; клон делается
  заново и удаляется, кэша нет
- добавлены init, add, pull, list, check в проекте — манифест
  .conventions.toml, сборка копий по разу на компонент, маркер локальной
  части, READING.md рядом
- проверки формы развязаны с набором: принимают lang.Vocabulary, а язык
  копии узнаётся по строке о версии — манифеста рядом с ней нет
This commit is contained in:
av
2026-07-27 20:42:18 +03:00
parent 4615de6e86
commit 23d88c4048
27 changed files with 3009 additions and 57 deletions
+55 -6
View File
@@ -33,9 +33,11 @@ CLI для управления конвенциями. Модель здесь
```
internal/lang словарь: реестр «версия языка × естественный язык»
internal/manifest suite.toml — чтение и текстовая правка
internal/source ссылки между уровнями: путь на диске, git-репозиторий
internal/manifest suite.toml и .conventions.toml — чтение и текстовая правка
internal/doc разбор документа: шапка, области правил, блоки
internal/suite сборка набора в память, отбор слоёв под компонент
internal/project сборка копий в проекте: разделы, маркер, READING.md
internal/check проверки: manifest, form, spread, links
internal/cli команды, диалог, два режима
```
@@ -67,6 +69,15 @@ internal/cli команды, диалог, два режима
первой в абзаце и полужирным. Огороженные блоки кода исключаются везде;
инлайн-код вырезается там, где ищутся ссылки, и не вырезается там, где ищутся
пути канона.
- **Уровень называется ссылкой, а не путём.** Проект ссылается на набор, набор
на язык; `source.Ref` разбирает ссылку, `source.Open` отдаёт директорию,
которую можно читать. Транспортов два, но `Kind` — перечисление, а не булево:
третий (rclone, дерево по https) ожидается, и отказ обязан сначала сказать,
чем ссылку сочли, и только потом — что не так.
- **Проверки формы не знают про набор.** `checkRules`, `checkVersionLine`,
`checkModalsOutside` и прочие принимают `lang.Vocabulary`, а не `*suite.Suite`
— иначе `convy check` в проекте пришлось бы писать заново. Копия несёт язык
строкой о версии, и `lang.Recognize` читает его оттуда: манифеста рядом нет.
## Решения, которые уже приняты
@@ -84,6 +95,26 @@ internal/cli команды, диалог, два режима
заголовков пропускаются.
- **Лок-файла, `push`, отчёта о расхождении и перенумерации не будет.** Модель
отвергает каждое явно.
- **Кэша источника нет.** Клон делается заново и удаляется вместе с `Tree`.
Кэш экономит второй клон и возвращает вопрос, что в нём протухло; на «что
было в прошлый раз» отвечает git потребителя. Если станет дорого, кэш
прячется за `source.Tree` и наружу не виден.
- **`file://` — это git, а не директория.** Простой путь уже означает «эта
директория, как она лежит», вместе с грязным рабочим деревом; `file://`
означает «тот же репозиторий в закоммиченном виде». Ради этой разницы оба
написания и существуют — и ради неё же git-транспорт тестируется без сети.
- **Слой ниже первого становится разделом.** Заголовки опускаются на уровень,
строка о версии языка выбрасывается у всех, кроме первого. Расширение
реализует и сужает базу, а не стоит рядом с ней, — поэтому правила базы в
копии на `###`, а правила языкового слоя на `####`. Проверка копий уровень
заголовка правила не требует, лестницу заголовков — требует.
- **`convy check` до набора не дотягивается.** Форма правила одна и та же,
локальные правила на `X` записаны по ней же, а проверять своё нужно без сети
и без знания, откуда копии приехали.
- **Целостность набора проверяется локально.** Когда `[language] source`
заполнен, `suite check` не тянет описание языка и пропускает проверки
документов о языке (META-30 в том числе), проверяя вместо этого саму ссылку.
Проверка гоняется на каждой правке и в сеть ходить не должна.
## Проверки
@@ -111,6 +142,16 @@ internal/cli команды, диалог, два режима
файл набора; упоминание `README.md` в конвенции она пометит ошибочно. На
текущем каноне не срабатывает.
- Машиночитаемого вывода находок (`--json`) нет.
- Предупреждения о висячей ссылке на неподписанную тему нет: `convy check` до
набора не дотягивается и разрешает только ссылки на префиксы самого файла.
Открытый вопрос из `TOOL.md`; закрывается отдельной командой, а не этой.
- Набор в поддиректории git-репозитория не адресуется: `#рев` есть, `//путь`
нет. Появится вместе с первым набором, который так лежит.
- Источник у проекта один. Модель нескольких допускает; форма `source = "..."`
расширяется до `[sources.имя]` не ломая существующие манифесты.
- `convy add` пишет подписку после сборки — если сборка прошла, а запись в
манифест упала, копия останется неучтённой. Обратный порядок хуже: подписка
без файла отправляет следующий `pull` искать то, чего не делали.
## Тесты
@@ -122,7 +163,12 @@ gofmt -l . && go vet ./... перед коммитом
Тесты фикстурные: набор пишется во временную директорию и прогоняется целиком.
`internal/cli` проверяет обе моды, включая диалог — интерактивный режим иначе не
покрыть, из шелла он требует терминала.
покрыть, из шелла он требует терминала. Проектные тесты строят набор теми же
командами и подключают его в проект: `subscribable` в `project_test.go`
`retirable` плюс `READING.md`, без которого копиям нечего везти рядом.
Git-транспорт проверяется на локальном репозитории через `file://` и
пропускается, если `git` не найден. Сети тесты не требуют.
## Коммиты
@@ -133,9 +179,12 @@ Commits и `Co-Authored-By` не используются.
## Состояние
Наборная сторона закончена: `init`, `add`, `rule`, `retire`, `list`, `check`.
Проектные команды (`add`, `pull`, `list`, `check` без `suite`) не начаты; отбор
слоёв под компонент для них уже написан — `suite.Assemble`, — и переписывать его
в сборщике не нужно.
Наборная сторона: `init`, `add`, `rule`, `retire`, `list`, `check`.
Проектная: `init`, `add`, `pull`, `list`, `check`. Обе стороны закончены по
тому, что намечено в `TOOL.md`.
Отбор слоёв под компонент — один на обе стороны: `suite.Assemble`. `suite list`
показывает, что взял бы компонент, `convy pull` то же самое пишет в файл;
разъехаться они не должны.
Линтеров и CI нет.
+114 -3
View File
@@ -12,7 +12,9 @@ CLI для управления конвенциями разработки: в
| Уровень | Что это |
|---|---|
| язык | как записывается правило: слова, версия, `READING.md` |
| набор | репозиторий с конвенциями, манифестом `suite.toml` и обвязкой |
| проект | репозиторий-потребитель с манифестом `.conventions.toml` |
| тема | набор правил об одном фокусе разработки; единица подписки |
| слой | один файл темы: базовый, языковой или стековый |
| компонент | адресат сборки в проекте: один язык, один стек, один вид приложения |
@@ -21,6 +23,37 @@ CLI для управления конвенциями разработки: в
Тему и ось слоя объявляет шапка файла, а не путь: `topic:`, `lang:`, `stack:`.
Слой без ключей оси — базовый, он попадает в копию всегда.
## Три уровня и ссылки между ними
Уровни стоят стопкой, и каждый нижний называет верхний ссылкой:
```
язык → слова, версия, короткое описание для читателя
набор → ссылается на язык: [language] version, lang, source
проект → ссылается на набор: source в .conventions.toml
```
Каждый уровень — набор файлов, а где эти файлы лежат, решает не модель, а
ссылка. Реализовано два транспорта:
| Ссылка | Что это |
|---|---|
| `../dev-conventions`, `/srv/conventions` | директория на диске, как она лежит |
| `https://git.example.org/av/conventions.git` | git-репозиторий, клонируется |
| `file:///srv/conventions#v2` | тот же репозиторий, но в закоммиченном виде |
Относительный путь считается от манифеста, который ссылку несёт. Хвост
`#ветка`, `#тег` или `#коммит` закрепляет ревизию и осмыслен только у git.
`ssh://` и `git@host:path` работают тем же клонированием, но проверены хуже.
Клон делается заново на каждый вызов и удаляется. Кэш сэкономил бы второй
клон и вернул бы вопрос, что в нём протухло, — а на «что было в прошлый раз»
отвечает git в репозитории-потребителе.
Ключ `[language] source` в наборе пока обычно пуст: описание языка живёт в
самом наборе. Когда спецификация уедет в свой репозиторий, тот же ключ её
назовёт, и больше ничего не изменится.
## Установка
Внешняя зависимость одна (`BurntSushi/toml`), сборка обычная:
@@ -48,10 +81,11 @@ go build -o convy .
convy suite list что в наборе и что возьмёт компонент
convy suite check целостность набора: префиксы, темы, оси, ссылки, форма
В проекте (пока не реализовано):
В проекте:
convy init подключить конвенции: источник и первый компонент
convy add <тема> подписаться и собрать
convy pull пересобрать подписанное
convy list что подключено и что доступно
convy list что подключено и что ещё есть в наборе
convy check проверить форму того, что здесь
```
@@ -118,6 +152,81 @@ errors: 1, warnings: 0
Коды возврата: `0` — чисто, `1` — есть ошибки, `2` — команда набрана неверно или
не в том контексте.
## Подключение в проект
```
$ convy init --source ../dev-conventions --component backend \
--dir backend/docs/conventions --lang go --stack sqlite
$ convy add time
$ convy add logging --for backend
```
`init` проверяет источник до того, как писать манифест: набор, до которого
никто не доберётся, проходит любую проверку и никому не помогает. Получается
`.conventions.toml`:
```toml
source = "../dev-conventions"
[components.backend]
dir = "backend/docs/conventions"
lang = ["go"]
stack = ["sqlite"]
topics = ["logging", "time"]
```
Компонент пишется всегда, даже когда он один; при нескольких команда без
`--for` не угадывает, а перечисляет имена. `stack` — список: `sqlite` и
`postgres` действуют вместе, это разные таблицы одного сервиса. Два языка не
действуют вместе никогда — ради этого компонент и заведён.
Копия плоская, файл на тему. Первый слой — сам документ; каждый следующий
становится его разделом, и заголовки внутри опускаются на уровень: слой
реализует и сужает базу, а не стоит рядом с ней. Строка о версии языка
остаётся одна.
```markdown
---
origin: time
---
# Время
...
### TIME-1. Единый формат — RFC 3339, UTC
...
## Время: реализация на Go
...
#### GTIM-1. «Сейчас» берётся у слоя хранилища
...
<!-- conv:local -->
```
Всё ниже `<!-- conv:local -->` принадлежит репозиторию и переживает `pull`;
всё выше перезаписывается. Рядом с копиями кладётся `READING.md` — он
приезжает с уровня языка. `README.md` в той же директории принадлежит проекту
и не трогается, а файл, у которого убрали `origin:`, перестаёт быть копией:
`pull` его не перезапишет и скажет почему.
## Проверка проекта
`convy check` до набора не дотягивается и сети не требует: форма правила одна
и та же, локальные правила проекта на `X`-префиксах записаны по ней же.
Проверяются шапка `origin:`, маркер локальной части, нумерация по каждому
префиксу, блоки правила, словарь — и то, чего в наборе не бывает:
```
$ convy check
docs/conventions/time.md
error:22 rule XTIM-1 is a rule of the repository standing above the
marker: a reassembly would wipe it [spread]
project: 2 files in 1 component
errors: 1, warnings: 0
```
Язык копии определяется по строке о версии, которую она несёт: манифест рядом
с ней не лежит, и больше сказать некому.
## Ступени и словарь
Слова, которыми записаны модальность и метки, — свойство версии языка и
@@ -144,4 +253,6 @@ errors: 1, warnings: 0
git;
- не хранит список подписчиков: подписка — свойство проекта;
- не переносит правки из проекта в набор: операция ручная и редкая;
- не перенумеровывает правила: номер — идентификатор, а не позиция.
- не перенумеровывает правила: номер — идентификатор, а не позиция;
- не кэширует источник: клон делается заново и удаляется;
- не знает нескольких наборов сразу: `source` в проекте один.
+181
View File
@@ -0,0 +1,181 @@
package check
import (
"sort"
"strings"
"git.vakhrushev.me/av/convy/internal/doc"
"git.vakhrushev.me/av/convy/internal/lang"
)
// Checking a copy is not checking a suite with parts left out. A copy has no
// manifest next to it, no prefix of its own and no path back to where it came
// from — it declares its language by the version line and its origin by one
// key, and that is everything a consuming repository holds.
//
// So what is checked here is what a copy answers for on its own: the form of a
// rule, which is the same form the suite writes, and the two things only a copy
// has — the marker of the local part, and the rule that whatever is written
// below it takes a prefix on X.
// CopyMarker is the boundary between what the suite wrote and what the
// repository wrote. It is passed in rather than known here: the marker belongs
// to the model of assembly, and the checks merely respect it.
const CopyMarker = "<!-- conv:local -->"
// Copy checks one assembled convention.
func Copy(d *doc.Document, rep *Report) {
if !checkOrigin(d, rep) {
return
}
v, ok := recognize(d, rep)
if !ok {
return
}
d.Blocks(v)
marker := markerLine(d)
checkMarker(d, marker, rep)
checkCopyHeadings(d, marker, rep)
checkHeadingHierarchy(d, rep)
for _, prefix := range prefixes(d) {
checkNumbering(d, prefix, rep)
}
checkRules(v, d, rep)
versionFrom, versionTo := checkVersionLine(v, d, rep)
checkModalsOutside(v, d, versionFrom, versionTo, rep)
checkForeignVocabulary(v, d, rep)
checkForeignConnectives(v, d, rep)
checkCopyLinks(d, rep)
}
// checkOrigin checks the front matter of a copy: one key, the name of the
// topic. The keys of a suite file have no business here — a copy is flat and
// carries no axis, and a stray extends would point at a path the repository
// does not have.
func checkOrigin(d *doc.Document, rep *Report) bool {
if !d.Front.Present || d.Front.Origin == "" {
rep.Errorf(Spread, d.Path, 1, "the file carries no origin key and is not a copy")
return false
}
for _, key := range []string{"topic", "prefix", "lang", "stack", "extends"} {
if at, ok := d.Front.At[key]; ok {
rep.Errorf(Spread, d.Path, at,
"the front matter of a copy carries the key %q of a suite file: a copy declares its topic by origin and nothing else", key)
}
}
for _, key := range d.Front.Unknown {
rep.Warnf(Spread, d.Path, d.Front.At[key], "front matter key %q is unknown to the tool", key)
}
return true
}
// recognize works out which vocabulary the copy is written in.
func recognize(d *doc.Document, rep *Report) (lang.Vocabulary, bool) {
from, to := d.Preamble()
var text []string
for _, p := range d.Paragraphs(from, to) {
text = append(text, p.Text())
}
v, ok := lang.Recognize(strings.Join(text, "\n"))
if !ok {
start, _ := d.Preamble()
rep.Errorf(Form, d.Path, start,
"the introductory prose holds no language version line, and it is the only thing that says which words of this file are normative")
return lang.Vocabulary{}, false
}
return v, true
}
// markerLine finds the line the local marker stands on, or zero.
func markerLine(d *doc.Document) int {
for n := 1; n <= d.Len(); n++ {
if strings.TrimSpace(d.Line(n)) == CopyMarker {
return n
}
}
return 0
}
func checkMarker(d *doc.Document, marker int, rep *Report) {
if marker == 0 {
rep.Errorf(Spread, d.Path, d.Len(),
"the copy carries no %s marker: there is nowhere to write a derogation, and a reassembly would overwrite whatever was written instead", CopyMarker)
return
}
for n := marker + 1; n <= d.Len(); n++ {
if strings.TrimSpace(d.Line(n)) == CopyMarker {
rep.Errorf(Spread, d.Path, n,
"the copy carries a second %s marker: the marker is one, and everything below the first belongs to the repository", CopyMarker)
return
}
}
}
// checkCopyHeadings checks what a rule heading of a copy answers for. The level
// is not among it: layers below the first become sections of the document when
// assembled, and their rules step down with them.
func checkCopyHeadings(d *doc.Document, marker int, rep *Report) {
for _, r := range d.Rules {
if r.Malformed != "" {
rep.Errorf(Form, d.Path, r.Line, "%s: %s", r.ID(), r.Malformed)
}
local := marker > 0 && r.Line > marker
switch {
case local && !strings.HasPrefix(r.Prefix, "X"):
rep.Errorf(Spread, d.Path, r.Line,
"rule %s stands below the marker and takes a prefix the suite could hand out: a rule of the repository takes a prefix on X", r.ID())
case !local && strings.HasPrefix(r.Prefix, "X"):
rep.Errorf(Spread, d.Path, r.Line,
"rule %s is a rule of the repository standing above the marker: a reassembly would wipe it", r.ID())
}
}
}
// prefixes lists the prefixes the rules of a copy use, in a stable order. There
// is more than one: a copy gathers the layers of a topic, and a layer brings its
// own prefix along.
func prefixes(d *doc.Document) []string {
seen := make(map[string]bool)
var out []string
for _, r := range d.Rules {
if !seen[r.Prefix] {
seen[r.Prefix] = true
out = append(out, r.Prefix)
}
}
sort.Strings(out)
return out
}
// checkCopyLinks resolves the references a copy can resolve: the ones to a
// prefix the file itself holds. A reference to another topic is left alone — the
// repository may well not be subscribed to it, and that is legitimate (META-20).
func checkCopyLinks(d *doc.Document, rep *Report) {
own := make(map[string]map[int]bool)
for _, r := range d.Rules {
if own[r.Prefix] == nil {
own[r.Prefix] = make(map[int]bool)
}
own[r.Prefix][r.Num] = true
}
for _, ref := range refsIn(d, d.Body, d.Len()) {
nums, mine := own[ref.Prefix]
if !mine || nums[ref.Num] {
continue
}
rep.Errorf(Links, d.Path, ref.Line,
"reference %s points at a rule this file does not hold, while it does hold the rules of %s",
ref.Text, ref.Prefix)
}
}
// Copies checks every copy handed to it.
func Copies(docs []*doc.Document) *Report {
rep := &Report{}
for _, d := range docs {
Copy(d, rep)
}
return rep
}
+26 -23
View File
@@ -20,11 +20,11 @@ func checkForm(s *suite.Suite, d *doc.Document, rep *Report) {
checkHeadings(d, prefix, rep)
checkHeadingHierarchy(d, rep)
checkNumbering(d, prefix, rep)
checkRules(s, d, rep)
versionFrom, versionTo := checkVersionLine(s, d, rep)
checkModalsOutside(s, d, versionFrom, versionTo, rep)
checkForeignVocabulary(s, d, rep)
checkForeignConnectives(s, d, rep)
checkRules(s.Vocab, d, rep)
versionFrom, versionTo := checkVersionLine(s.Vocab, d, rep)
checkModalsOutside(s.Vocab, d, versionFrom, versionTo, rep)
checkForeignVocabulary(s.Vocab, d, rep)
checkForeignConnectives(s.Vocab, d, rep)
}
// checkHeadingHierarchy checks the ladder of headings: one title, and no level
@@ -62,8 +62,8 @@ func checkHeadingHierarchy(d *doc.Document, rep *Report) {
// vocabulary. Only the start of a line counts: that is where a scenario block
// puts them, while mid-sentence AND and OR belong to SQL far more often than to
// a mixture of vocabularies.
func checkForeignConnectives(s *suite.Suite, d *doc.Document, rep *Report) {
foreign := lang.ForeignConnectives(s.Manifest.Language.Version, s.Manifest.Language.Lang)
func checkForeignConnectives(v lang.Vocabulary, d *doc.Document, rep *Report) {
foreign := lang.ForeignConnectives(v.Version, v.Code)
if len(foreign) == 0 {
return
}
@@ -80,8 +80,8 @@ func checkForeignConnectives(s *suite.Suite, d *doc.Document, rep *Report) {
continue
}
rep.Errorf(Form, d.Path, n,
"the scenario block opens with %s from the %q vocabulary, while the suite declares %q",
w, foreign[w], s.Manifest.Language.Lang)
"the scenario block opens with %s from the %q vocabulary, while the document is written in %q",
w, foreign[w], v.Code)
break
}
return true
@@ -142,10 +142,14 @@ func checkHeadings(d *doc.Document, prefix string, rep *Report) {
// why there is never one, and a retired rule stays as a stub.
func checkNumbering(d *doc.Document, prefix string, rep *Report) {
seen := make(map[int][]int)
first := 0
for _, r := range d.Rules {
if r.Prefix != prefix {
continue
}
if first == 0 {
first = r.Line
}
seen[r.Num] = append(seen[r.Num], r.Line)
}
if len(seen) == 0 {
@@ -173,7 +177,7 @@ func checkNumbering(d *doc.Document, prefix string, rep *Report) {
}
}
if len(gaps) > 0 {
rep.Errorf(Form, d.Path, d.Rules[0].Line,
rep.Errorf(Form, d.Path, first,
"numbering is not contiguous: the highest number is %d, missing %s — a retired rule stays as a stub instead of disappearing",
highest, joinInts(gaps))
}
@@ -184,8 +188,7 @@ var dateRe = regexp.MustCompile(`\d{4}-\d{2}-\d{2}`)
// checkRules checks what a rule is made of: either a norm with a rationale, or
// the stub of a retired one. Neither the norm nor the rationale is ever deleted
// (META-8, META-10).
func checkRules(s *suite.Suite, d *doc.Document, rep *Report) {
v := s.Vocab
func checkRules(v lang.Vocabulary, d *doc.Document, rep *Report) {
for _, r := range d.Rules {
if retired, ok := r.Block(lang.Retired); ok {
checkRetired(d, r, retired, rep)
@@ -255,15 +258,15 @@ func checkRetired(d *doc.Document, r doc.Rule, retired doc.Block, rep *Report) {
// The line lists the key words of the suite and carries the rule of capitals
// itself — which makes it the only place outside rules where modal words are
// lawful.
func checkVersionLine(s *suite.Suite, d *doc.Document, rep *Report) (from, to int) {
p, ok := versionParagraph(s, d)
func checkVersionLine(v lang.Vocabulary, d *doc.Document, rep *Report) (from, to int) {
p, ok := versionParagraph(v, d)
if !ok {
start, _ := d.Preamble()
rep.Errorf(Form, d.Path, start,
"the introductory prose holds no language version line: it lists the key words of the suite, and without it a convention in a foreign repository loses the key to its own text")
return 0, 0
}
version := strconv.Itoa(s.Manifest.Language.Version)
version := strconv.Itoa(v.Version)
if !containsNumber(p.Text(), version) {
rep.Errorf(Form, d.Path, p.Start,
"the language version line does not name version %s declared by the suite manifest", version)
@@ -275,9 +278,9 @@ func checkVersionLine(s *suite.Suite, d *doc.Document, rep *Report) (from, to in
// the language version line: the one listing every key word of the suite. It
// reports nothing — checkVersionLine speaks about its absence, and speaking
// twice helps no one.
func versionParagraph(s *suite.Suite, d *doc.Document) (doc.Paragraph, bool) {
func versionParagraph(v lang.Vocabulary, d *doc.Document) (doc.Paragraph, bool) {
from, to := d.Preamble()
words := s.Vocab.Words()
words := v.Words()
for _, p := range d.Paragraphs(from, to) {
if containsAll(p.Text(), words) {
return p, true
@@ -289,8 +292,8 @@ func versionParagraph(s *suite.Suite, d *doc.Document) (doc.Paragraph, bool) {
// checkModalsOutside looks for capitalized modal words outside rule areas. An
// area runs from the heading of a rule to the next heading; everything else is
// prose, and prose is never a norm.
func checkModalsOutside(s *suite.Suite, d *doc.Document, versionFrom, versionTo int, rep *Report) {
words := modalWords(s.Vocab)
func checkModalsOutside(v lang.Vocabulary, d *doc.Document, versionFrom, versionTo int, rep *Report) {
words := modalWords(v)
d.Prose(func(n int, text string) bool {
if d.InRule(n) || n >= versionFrom && n <= versionTo {
return true
@@ -310,8 +313,8 @@ func checkModalsOutside(s *suite.Suite, d *doc.Document, versionFrom, versionTo
// checkForeignVocabulary looks for words of another vocabulary of the same
// language version. There is one vocabulary per suite: two ways of writing the
// same requirement double every check.
func checkForeignVocabulary(s *suite.Suite, d *doc.Document, rep *Report) {
foreign := lang.Foreign(s.Manifest.Language.Version, s.Manifest.Language.Lang)
func checkForeignVocabulary(v lang.Vocabulary, d *doc.Document, rep *Report) {
foreign := lang.Foreign(v.Version, v.Code)
if len(foreign) == 0 {
return
}
@@ -325,8 +328,8 @@ func checkForeignVocabulary(s *suite.Suite, d *doc.Document, rep *Report) {
for _, w := range words {
if containsWord(text, w) {
rep.Errorf(Form, d.Path, n,
"the word %s belongs to the %q vocabulary, while the suite declares %q",
w, foreign[w], s.Manifest.Language.Lang)
"the word %s belongs to the %q vocabulary, while the document is written in %q",
w, foreign[w], v.Code)
}
}
return true
+1 -1
View File
@@ -155,7 +155,7 @@ func checkMechanized(s *suite.Suite, d *doc.Document, rep *Report) {
if word == "" {
return
}
version, hasVersion := versionParagraph(s, d)
version, hasVersion := versionParagraph(s.Vocab, d)
d.Prose(func(n int, text string) bool {
if hasVersion && n >= version.Start && n <= version.End {
return true
+31 -3
View File
@@ -8,6 +8,7 @@ import (
"strings"
"git.vakhrushev.me/av/convy/internal/manifest"
"git.vakhrushev.me/av/convy/internal/source"
"git.vakhrushev.me/av/convy/internal/suite"
)
@@ -15,6 +16,7 @@ import (
func Suite(s *suite.Suite) *Report {
rep := &Report{}
checkManifest(s, rep)
checkLanguageSource(s, rep)
checkTopicNames(s, rep)
checkBaseLayers(s, rep)
checkSelfGoverning(s, rep)
@@ -39,9 +41,15 @@ func checkManifest(s *suite.Suite, rep *Report) {
for _, key := range m.Undecoded {
rep.Warnf(Manifest, manifest.Name, 0, "the key %s is unknown to the tool", key)
}
for _, name := range []string{m.Language.Description, m.Language.Reading} {
if name != "" && !s.Exists(name) {
rep.Errorf(Manifest, manifest.Name, 0, "the [language] section declares the document %q, and the file is missing", name)
// The documents about the language are named relative to the level they
// belong to. While that level is the suite, the files have to be here; once
// the language lives apart, they are out of reach of a check that runs on
// every edit and must not touch the network.
if m.Language.Source == "" {
for _, name := range []string{m.Language.Description, m.Language.Reading} {
if name != "" && !s.Exists(name) {
rep.Errorf(Manifest, manifest.Name, 0, "the [language] section declares the document %q, and the file is missing", name)
}
}
}
@@ -164,6 +172,12 @@ func checkSelfGoverning(s *suite.Suite, rep *Report) {
// fails in silence.
func checkReadingVocabulary(s *suite.Suite, rep *Report) {
name := s.Manifest.Language.Reading
if s.Manifest.Language.Source != "" {
// The account lies at a level of its own, and reaching it costs a
// fetch. Checking the integrity of a suite runs on every edit, so it
// stays local; the reference itself is checked instead.
return
}
if name == "" {
// A suite that has not written the document yet; `suite init` says so
// among the next steps, and repeating it on every run is noise.
@@ -218,3 +232,17 @@ func sortedKeys[V any](m map[string]V) []string {
sort.Strings(keys)
return keys
}
// checkLanguageSource checks the reference to the level of the language. What
// it names cannot be reached without a fetch, and a check of a suite does not
// fetch — but a reference that means nothing is caught here rather than at the
// first assembly in a foreign repository.
func checkLanguageSource(s *suite.Suite, rep *Report) {
raw := s.Manifest.Language.Source
if raw == "" {
return
}
if _, err := source.Parse(raw); err != nil {
rep.Errorf(Manifest, manifest.Name, 0, "[language] source: %s", err)
}
}
+168
View File
@@ -0,0 +1,168 @@
package cli
import (
"flag"
"fmt"
"os"
"strings"
"git.vakhrushev.me/av/convy/internal/manifest"
"git.vakhrushev.me/av/convy/internal/project"
)
// convy add does two things at once, and they are one thing: it writes the
// subscription into the manifest and assembles the file. A subscription that
// left no file behind, or a file nothing subscribed to, is the state the model
// has no name for.
func runAdd(env Env, args []string) ExitCode {
fs := flag.NewFlagSet("convy add", flag.ContinueOnError)
fs.SetOutput(env.Err)
root := fs.String("root", "", "root of the project; it is looked up upwards by default")
forComponent := fs.String("for", "", "component to assemble for; needed when there is more than one")
topicFlag := fs.String("topic", "", "topic to subscribe to; may also be given as the first argument")
// The topic is taken off the front before the flags are parsed. The flag
// package stops at the first argument that is not a flag, so "convy add
// time --for backend" — the form the model writes — would otherwise lose
// the component silently.
topic := ""
rest := args
if len(rest) > 0 && !strings.HasPrefix(rest[0], "-") {
topic, rest = rest[0], rest[1:]
}
if err := fs.Parse(rest); err != nil {
return Usage
}
if *topicFlag != "" {
if topic != "" && topic != *topicFlag {
fmt.Fprintf(env.Err, "the topic is named twice and differently: %q and %q\n", topic, *topicFlag)
return Usage
}
topic = *topicFlag
}
if rest := fs.Args(); len(rest) > 0 {
if topic != "" && topic != rest[0] {
fmt.Fprintf(env.Err, "the topic is named twice and differently: %q and %q\n", topic, rest[0])
return Usage
}
topic = rest[0]
}
o, code := openProject(env, *root)
if code != OK {
return code
}
defer o.Close()
if err := distinctDirs(o.Manifest); err != nil {
fmt.Fprintln(env.Err, err)
return Usage
}
name, c, code := componentOf(env, o.Manifest, *forComponent)
if code != OK {
return code
}
if topic == "" {
if !env.Interactive {
fmt.Fprintln(env.Err, "convy add without arguments asks which topic, and there is no terminal to ask on; name the topic as an argument")
return Usage
}
answer, err := newDialogue(env).ask(topicField(o, c))
if err != nil {
fmt.Fprintln(env.Err, "\ninterrupted, nothing was written")
return Usage
}
topic = answer
}
if !o.Suite.Manifest.TopicLive(topic) {
if o.Suite.Manifest.TopicRetired(topic) {
fmt.Fprintf(env.Err, "the suite has retired the topic %q: %s\n", topic, o.Suite.Manifest.Topics.Retired[topic])
return Usage
}
fmt.Fprintf(env.Err, "the suite declares no topic %q; it declares: %s\n", topic, strings.Join(o.Suite.Manifest.LiveTopics(), ", "))
return Usage
}
if c.Subscribed(topic) {
fmt.Fprintf(env.Err, "the component %q is subscribed to %q already; convy pull reassembles it\n", name, topic)
return Usage
}
made, err := project.Assemble(o.Suite, o.Root, c, topic)
if err != nil {
fmt.Fprintln(env.Err, err)
return Failed
}
// The manifest is written after the file: a subscription recorded against
// an assembly that failed would send the next pull looking for a copy that
// was never made.
source, err := os.ReadFile(o.Manifest.Path)
if err != nil {
fmt.Fprintln(env.Err, err)
return Failed
}
updated, err := manifest.AddToList(source, "components."+name, "topics", topic)
if err != nil {
fmt.Fprintln(env.Err, err)
return Failed
}
if err := os.WriteFile(o.Manifest.Path, updated, 0o644); err != nil {
fmt.Fprintln(env.Err, err)
return Failed
}
guide, err := project.Reading(o.LangRoot, o.Suite.Manifest.Language.Reading, o.Root, c.Dir)
if err != nil {
fmt.Fprintf(env.Err, "warning: %s\n", err)
}
fmt.Fprintf(env.Out, "\n%s → %s\n", topic, made.Path)
for _, layer := range made.Layers {
fmt.Fprintf(env.Out, " %s\n", layer)
}
if guide != "" {
fmt.Fprintf(env.Out, "\n%s refreshed\n", guide)
}
fmt.Fprintf(env.Out, "subscribed the component %q in %s\n", name, manifest.ProjectName)
return OK
}
// topicField offers the topics the component has not taken yet.
func topicField(o *opened, c manifest.Component) Field {
var free []string
for _, t := range o.Suite.Manifest.LiveTopics() {
if !c.Subscribed(t) {
free = append(free, t)
}
}
return Field{
Flag: "topic",
Ask: "Topic",
Hint: "A topic is taken whole: the file gathers every layer of it the component fits. " + describeTopics(o, free),
Options: free,
Check: func(v string) error {
if !o.Suite.Manifest.TopicLive(v) {
return fmt.Errorf("the suite declares no topic %q", v)
}
if c.Subscribed(v) {
return fmt.Errorf("this component is subscribed to %q already", v)
}
return nil
},
}
}
func describeTopics(o *opened, free []string) string {
if len(free) == 0 {
return "The component is subscribed to everything the suite has."
}
parts := make([]string, 0, len(free))
for _, t := range free {
parts = append(parts, fmt.Sprintf("%s — %s", t, o.Suite.Manifest.Topics.Live[t]))
}
return "On offer: " + strings.Join(parts, "; ") + "."
}
+142
View File
@@ -0,0 +1,142 @@
package cli
import (
"flag"
"fmt"
"io"
"io/fs"
"os"
"path/filepath"
"sort"
"git.vakhrushev.me/av/convy/internal/check"
"git.vakhrushev.me/av/convy/internal/doc"
"git.vakhrushev.me/av/convy/internal/manifest"
"git.vakhrushev.me/av/convy/internal/project"
)
// convy check stays at the top level and reaches for no suite. The form of a
// rule is one and the same, the local rules of the repository on X prefixes are
// written by that same form, and checking what lies here has to work without a
// network and without knowing where the copies came from.
func runCheck(env Env, args []string) ExitCode {
fs := flag.NewFlagSet("convy check", flag.ContinueOnError)
fs.SetOutput(env.Err)
root := fs.String("root", "", "root of the project; it is looked up upwards by default")
forComponent := fs.String("for", "", "component to check; every one of them by default")
quiet := fs.Bool("quiet", false, "print findings only")
if err := fs.Parse(args); err != nil {
return Usage
}
dir, code := projectRoot(env, *root)
if code != OK {
return code
}
m, err := manifest.LoadProject(dir)
if err != nil {
fmt.Fprintln(env.Err, err)
return Usage
}
names, code := components(env, m, *forComponent)
if code != OK {
return code
}
var docs []*doc.Document
var broken []error
for _, name := range names {
found, errs := copies(dir, m.Components[name].Dir)
docs = append(docs, found...)
broken = append(broken, errs...)
}
rep := check.Copies(docs)
for _, err := range broken {
fmt.Fprintf(env.Err, "%s\n", err)
}
printCopyReport(env.Out, rep, len(docs), len(names), *quiet)
if rep.Errors() > 0 || len(broken) > 0 {
return Failed
}
return OK
}
// copies collects the assembled conventions of one component directory.
//
// What is a copy is decided by the origin key rather than by the name of the
// file: README.md belongs to the repository, READING.md belongs to the suite,
// and a file whose origin key was taken away has become a document of the
// repository — none of the three answers to the form of a rule.
func copies(root, dir string) ([]*doc.Document, []error) {
var docs []*doc.Document
var broken []error
base := filepath.Join(root, filepath.FromSlash(dir))
err := filepath.WalkDir(base, func(name string, entry fs.DirEntry, err error) error {
if err != nil {
return err
}
if entry.IsDir() || filepath.Ext(entry.Name()) != ".md" {
return nil
}
rel, err := filepath.Rel(root, name)
if err != nil {
return err
}
rel = filepath.ToSlash(rel)
d, err := doc.Load(rel, name)
if err != nil {
broken = append(broken, err)
return nil
}
if d.Front.Origin == "" {
return nil
}
docs = append(docs, d)
return nil
})
if err != nil && !os.IsNotExist(err) {
broken = append(broken, fmt.Errorf("walking %s: %w", dir, err))
}
sort.Slice(docs, func(i, j int) bool { return docs[i].Path < docs[j].Path })
return docs, broken
}
func printCopyReport(w io.Writer, rep *check.Report, files, comps int, quiet bool) {
findings := rep.Findings()
current := ""
for _, f := range findings {
if f.Path != current {
if current != "" {
fmt.Fprintln(w)
}
fmt.Fprintf(w, "%s\n", f.Path)
current = f.Path
}
where := ""
if f.Line > 0 {
where = fmt.Sprintf(":%d", f.Line)
}
fmt.Fprintf(w, " %s%s %s [%s]\n", f.Severity, where, f.Msg, f.Family)
}
if quiet {
return
}
if len(findings) > 0 {
fmt.Fprintln(w)
}
fmt.Fprintf(w, "project: %s in %s\n", plural(files, "file"), plural(comps, "component"))
switch {
case rep.Errors() > 0:
fmt.Fprintf(w, "errors: %d, warnings: %d\n", rep.Errors(), rep.Warnings())
case rep.Warnings() > 0:
fmt.Fprintf(w, "no errors, warnings: %d\n", rep.Warnings())
case files == 0:
fmt.Fprintf(w, "nothing to check: no file carries an origin key; convy pull assembles the copies\n")
default:
fmt.Fprintf(w, "the copies hold the form; everything below %s is the repository's own\n", project.LocalMarker)
}
}
+15 -7
View File
@@ -48,9 +48,16 @@ func Run(env Env, args []string) ExitCode {
switch args[0] {
case "suite":
return runSuite(env, args[1:])
case "add", "pull", "list", "check":
fmt.Fprintf(env.Err, "the %q command is not implemented yet\n", args[0])
return Usage
case "init":
return runInit(env, args[1:])
case "add":
return runAdd(env, args[1:])
case "pull":
return runPull(env, args[1:])
case "list":
return runList(env, args[1:])
case "check":
return runCheck(env, args[1:])
case "help", "-h", "--help":
usage(env.Out)
return OK
@@ -90,10 +97,11 @@ func usage(w io.Writer) {
fmt.Fprint(w, `convy — tending development conventions.
In a project:
convy add <topic> subscribe and assemble (not implemented)
convy pull reassemble what is subscribed (not implemented)
convy list what is wired up and available (not implemented)
convy check check the form of what is here (not implemented)
convy init wire up conventions: the source and the first component
convy add <topic> subscribe and assemble
convy pull reassemble what is subscribed
convy list what is wired up and what else the suite has
convy check check the form of what is here
In a suite:
convy suite init start a suite: a directory and a manifest
+192
View File
@@ -0,0 +1,192 @@
package cli
import (
"flag"
"fmt"
"os"
"path/filepath"
"strings"
"git.vakhrushev.me/av/convy/internal/manifest"
"git.vakhrushev.me/av/convy/internal/source"
"git.vakhrushev.me/av/convy/internal/suite"
)
// projectSkeleton is what a project manifest starts as. It is written as text
// with its comments in place for the same reason the suite manifest is: the
// file is read far more often than a tool touches it, and what a component is
// for is not deducible from three keys.
const projectSkeleton = `# What this repository takes from a conventions suite.
#
# Two manifests exist in the model, each named after what it describes:
# suite.toml in a suite describes the suite itself; .conventions.toml here
# describes the subscription — where the copies come from and who takes what.
#
# Copies are committed. Nothing is fetched on the fly, and the answer to "what
# did it look like last time" is given by git rather than by a lock file.
# Where the copies come from: a path on disk, relative to this file or absolute,
# or a git repository over http or https. A trailing #branch, #tag or #commit
# pins a revision.
source = "%s"
# ─── Components ─────────────────────────────────────────────────────────────
#
# A component is a region of the repository where every selected layer holds at
# once: one language, one set of tools, one kind of application. Each one has
# its own directory, and the directories differ — that is the only thing that
# tells two copies of one topic apart.
#
# lang and stack choose the layers: a layer travels when the axis it declares is
# among them, and a layer declaring no axis travels always. topics is the
# subscription itself, by the names the suite gives its topics.
`
func runInit(env Env, args []string) ExitCode {
fs := flag.NewFlagSet("convy init", flag.ContinueOnError)
fs.SetOutput(env.Err)
root := fs.String("root", "", "root of the project; the current directory by default")
from := fs.String("source", "", "reference to the suite: a path on disk or a git repository")
component := fs.String("component", "", "name of the first component")
dir := fs.String("dir", "", "directory the copies of that component go into")
langAxis := fs.String("lang", "", "language of the component")
stackAxis := fs.String("stack", "", "stack of the component, comma-separated")
if err := fs.Parse(args); err != nil {
return Usage
}
where := *root
if where == "" {
where = env.Dir
}
if exists(filepath.Join(where, manifest.ProjectName)) {
fmt.Fprintf(env.Err, "%s already holds %s: the conventions are wired up already\n", where, manifest.ProjectName)
fmt.Fprintln(env.Err, "convy add subscribes to one more topic")
return Usage
}
given := map[string]string{
"source": *from, "component": *component, "dir": *dir,
"lang": *langAxis, "stack": *stackAxis,
}
if len(args) == 0 {
if !env.Interactive {
fmt.Fprintln(env.Err, "convy init without arguments asks questions, and there is no terminal to ask on; pass --source, --component and --dir")
return Usage
}
if err := askAll(env, initFields(), given); err != nil {
return Usage
}
} else if err := resolve(initFields(), given); err != nil {
fmt.Fprintln(env.Err, err)
return Usage
}
// The suite is reached before the manifest is written. A manifest naming a
// suite nobody can reach passes every check the tool has and helps no one.
ref, err := source.Parse(given["source"])
if err != nil {
fmt.Fprintln(env.Err, err)
return Usage
}
tree, err := source.Open(ref, where)
if err != nil {
fmt.Fprintln(env.Err, err)
return Failed
}
defer tree.Close()
s, err := suite.Load(tree.Dir())
if err != nil {
fmt.Fprintf(env.Err, "the source %s is not a conventions suite: %s\n", ref, tree.Describe(err))
return Failed
}
entries := [][2]string{{"dir", quote(given["dir"])}}
if list := split(given["lang"]); len(list) > 0 {
entries = append(entries, [2]string{"lang", array(list)})
}
if list := split(given["stack"]); len(list) > 0 {
entries = append(entries, [2]string{"stack", array(list)})
}
entries = append(entries, [2]string{"topics", "[]"})
body, err := manifest.AddTable([]byte(fmt.Sprintf(projectSkeleton, given["source"])),
"components."+given["component"], entries)
if err != nil {
fmt.Fprintln(env.Err, err)
return Failed
}
if err := os.MkdirAll(where, 0o755); err != nil {
fmt.Fprintln(env.Err, err)
return Failed
}
name := filepath.Join(where, manifest.ProjectName)
if err := os.WriteFile(name, body, 0o644); err != nil {
fmt.Fprintln(env.Err, err)
return Failed
}
fmt.Fprintf(env.Out, "\ncreated %s\n", name)
fmt.Fprintf(env.Out, "the suite speaks %s, conventions language version %d\n",
s.Manifest.Language.Lang, s.Manifest.Language.Version)
topics := s.Manifest.LiveTopics()
if len(topics) > 0 {
fmt.Fprintf(env.Out, "\n%s to take from:\n", plural(len(topics), "topic"))
for _, t := range topics {
fmt.Fprintf(env.Out, " %-20s %s\n", t, s.Manifest.Topics.Live[t])
}
}
fmt.Fprint(env.Out, `
next:
convy add <topic> subscribe and assemble
convy list what is wired up and what else is there
`)
return OK
}
func initFields() []Field {
return []Field{{
Flag: "source",
Ask: "Where the copies come from",
Hint: "A path to the suite on disk — relative to this repository or absolute — or a git repository over http or https. A trailing #branch, #tag or #commit pins a revision.",
}, {
Flag: "component",
Ask: "Name of the component",
Hint: "A region of the repository where all the chosen layers hold at once: one language, one set of tools. The name is not internal — it is how the tool answers what it assembled and where.",
Check: func(v string) error {
if !nameRe.MatchString(v) {
return fmt.Errorf("a component is named by a plain identifier: letters, digits, a dash")
}
return nil
},
}, {
Flag: "dir",
Ask: "Directory of the copies",
Hint: "Where the assembled files go. Every component has its own, and two components never share one: the copies of a topic would collide by name.",
Default: "docs/conventions",
}, {
Flag: "lang",
Ask: "Language of the component",
Hint: "Chooses the language layers. Empty when the suite is flat and has no axes at all.",
Optional: true,
}, {
Flag: "stack",
Ask: "Stack of the component",
Hint: "Chooses the stack layers — the storage, the transport, the tools. Several are allowed, comma-separated: sqlite and postgres hold together, being different tables of one service.",
Optional: true,
}}
}
func quote(s string) string {
return `"` + strings.ReplaceAll(s, `"`, `\"`) + `"`
}
func array(values []string) string {
parts := make([]string, len(values))
for i, v := range values {
parts[i] = quote(v)
}
return "[" + strings.Join(parts, ", ") + "]"
}
+108
View File
@@ -0,0 +1,108 @@
package cli
import (
"flag"
"fmt"
"path/filepath"
"strings"
"git.vakhrushev.me/av/convy/internal/manifest"
"git.vakhrushev.me/av/convy/internal/project"
)
// convy list answers two questions in one view: what this repository takes and
// what the suite has that it does not. The second half is the reason the
// command reaches the suite at all — a listing of the manifest alone is the
// manifest, and reading it needs no tool.
func runList(env Env, args []string) ExitCode {
fs := flag.NewFlagSet("convy list", flag.ContinueOnError)
fs.SetOutput(env.Err)
root := fs.String("root", "", "root of the project; it is looked up upwards by default")
forComponent := fs.String("for", "", "component to show; every one of them by default")
if err := fs.Parse(args); err != nil {
return Usage
}
o, code := openProject(env, *root)
if code != OK {
return code
}
defer o.Close()
names, code := components(env, o.Manifest, *forComponent)
if code != OK {
return code
}
fmt.Fprintf(env.Out, "source %s\n", o.Manifest.Source)
fmt.Fprintf(env.Out, "%s, language version %d (%s)\n",
plural(len(o.Suite.Manifest.LiveTopics()), "topic"),
o.Suite.Manifest.Language.Version, o.Suite.Manifest.Language.Lang)
for _, name := range names {
c := o.Manifest.Components[name]
fmt.Fprintf(env.Out, "\n%s — %s%s\n", name, c.Dir, axisOf(c))
listComponent(env, o, c)
}
return OK
}
func listComponent(env Env, o *opened, c manifest.Component) {
if len(c.Topics) == 0 {
fmt.Fprintln(env.Out, " subscribed to nothing yet")
}
for _, topic := range c.Topics {
if !o.Suite.Manifest.TopicLive(topic) {
fmt.Fprintf(env.Out, " %-20s the suite declares no such topic any more\n", topic)
continue
}
taken, _ := o.Suite.Assemble(topic, project.Axis(c))
file := filepath.ToSlash(filepath.Join(c.Dir, topic+".md"))
mark := " "
if !exists(filepath.Join(o.Root, filepath.FromSlash(file))) {
mark = "!"
}
fmt.Fprintf(env.Out, "%s %-20s %-16s %s\n", mark, topic, plural(len(taken), "layer"),
o.Suite.Manifest.Topics.Live[topic])
if mark == "!" {
fmt.Fprintf(env.Out, " %-20s subscribed, and no file: convy pull assembles it\n", "")
}
}
var free []string
for _, topic := range o.Suite.Manifest.LiveTopics() {
if !c.Subscribed(topic) {
free = append(free, topic)
}
}
if len(free) == 0 {
return
}
fmt.Fprintln(env.Out, "\n not taken:")
for _, topic := range free {
taken, _ := o.Suite.Assemble(topic, project.Axis(c))
if len(taken) == 0 {
fmt.Fprintf(env.Out, "· %-20s %-16s %s\n", topic, "no layer fits",
o.Suite.Manifest.Topics.Live[topic])
continue
}
fmt.Fprintf(env.Out, "· %-20s %-16s %s\n", topic, plural(len(taken), "layer"),
o.Suite.Manifest.Topics.Live[topic])
}
}
// axisOf describes the axis of a component the way the suite writes it.
func axisOf(c manifest.Component) string {
var parts []string
if len(c.Lang) > 0 {
parts = append(parts, "lang="+strings.Join(c.Lang, ","))
}
if len(c.Stack) > 0 {
parts = append(parts, "stack="+strings.Join(c.Stack, ","))
}
if len(parts) == 0 {
return ""
}
return " (" + strings.Join(parts, " ") + ")"
}
+167
View File
@@ -0,0 +1,167 @@
package cli
import (
"fmt"
"os"
"path/filepath"
"git.vakhrushev.me/av/convy/internal/manifest"
"git.vakhrushev.me/av/convy/internal/source"
"git.vakhrushev.me/av/convy/internal/suite"
)
// A project command works against two levels at once: the manifest lying in the
// repository and the suite it names. Reaching the suite costs a clone when the
// reference is a git one, so it is done once per command and released at the
// end — nothing of the suite is left lying about in the project.
// opened is a project together with the levels above it.
type opened struct {
Root string
Manifest *manifest.Project
Suite *suite.Suite
// LangRoot is where the documents about the language lie. It is the root
// of the suite while the language has no repository of its own.
LangRoot string
trees []*source.Tree
}
// Close releases whatever the opening fetched.
func (o *opened) Close() {
for _, t := range o.trees {
t.Close()
}
}
// openProject finds the project, resolves its source and loads the suite.
func openProject(env Env, root string) (*opened, ExitCode) {
dir, code := projectRoot(env, root)
if code != OK {
return nil, code
}
m, err := manifest.LoadProject(dir)
if err != nil {
fmt.Fprintln(env.Err, err)
return nil, Usage
}
for _, key := range m.Undecoded {
fmt.Fprintf(env.Err, "warning: %s: the key %s is unknown to the tool\n", m.Path, key)
}
if m.Source == "" {
fmt.Fprintf(env.Err, "%s names no source: a copy comes from a suite, and the manifest is where the suite is named\n", m.Path)
return nil, Usage
}
ref, err := source.Parse(m.Source)
if err != nil {
fmt.Fprintln(env.Err, err)
return nil, Usage
}
tree, err := source.Open(ref, dir)
if err != nil {
fmt.Fprintln(env.Err, err)
return nil, Failed
}
o := &opened{Root: dir, Manifest: m, LangRoot: tree.Dir(), trees: []*source.Tree{tree}}
o.Suite, err = suite.Load(tree.Dir())
if err != nil {
fmt.Fprintf(env.Err, "the source %s is not a conventions suite: %s\n", ref, tree.Describe(err))
o.Close()
return nil, Failed
}
// The language is a level of its own, and a suite may keep its documents
// apart from itself. While it does not, the suite is where they lie.
if spec := o.Suite.Manifest.Language.Source; spec != "" {
ref, err := source.Parse(spec)
if err != nil {
o.Close()
fmt.Fprintf(env.Err, "%s: [language] source: %s\n", o.Suite.Manifest.Path, err)
return nil, Usage
}
langTree, err := source.Open(ref, tree.Dir())
if err != nil {
o.Close()
fmt.Fprintln(env.Err, err)
return nil, Failed
}
o.trees = append(o.trees, langTree)
o.LangRoot = langTree.Dir()
}
return o, OK
}
// projectRoot finds the manifest of the project. A project command typed inside
// a suite does not do anything at a guess: it says where it is and names the
// command of that level.
func projectRoot(env Env, given string) (string, ExitCode) {
if given != "" {
return given, OK
}
found, err := manifest.FindProject(env.Dir)
if err == nil {
return found, OK
}
if _, suiteErr := manifest.Find(env.Dir); suiteErr == nil {
fmt.Fprintf(env.Err, "this is a conventions suite, not a project that takes copies: %s lies here, %s does not\n", manifest.Name, manifest.ProjectName)
fmt.Fprintln(env.Err, "the commands of a suite are under convy suite")
return "", Usage
}
fmt.Fprintf(env.Err, "not a project with conventions: no %s here or above\n", manifest.ProjectName)
fmt.Fprintln(env.Err, "convy init wires one up")
return "", Usage
}
// componentOf resolves the --for flag against the manifest.
func componentOf(env Env, m *manifest.Project, name string) (string, manifest.Component, ExitCode) {
got, c, err := m.Only(name)
if err != nil {
fmt.Fprintln(env.Err, err)
return "", manifest.Component{}, Usage
}
if c.Dir == "" {
fmt.Fprintf(env.Err, "the component %q names no dir, and a copy has to be written somewhere\n", got)
return "", manifest.Component{}, Usage
}
return got, c, OK
}
// components picks the components a command works on: the one named by --for,
// or every one of them.
func components(env Env, m *manifest.Project, name string) ([]string, ExitCode) {
if name != "" {
if _, ok := m.Components[name]; !ok {
fmt.Fprintf(env.Err, "the project declares no component %q\n", name)
return nil, Usage
}
return []string{name}, OK
}
if len(m.Components) == 0 {
fmt.Fprintf(env.Err, "%s declares no component, and a copy is assembled for a component\n", m.Path)
return nil, Usage
}
return m.Names(), OK
}
// distinctDirs checks that no two components write into the same directory.
// Two copies of one topic would otherwise collide by name, and that is an error
// of the manifest rather than a reason to rename files.
func distinctDirs(m *manifest.Project) error {
seen := make(map[string]string)
for _, name := range m.Names() {
dir := filepath.ToSlash(filepath.Clean(m.Components[name].Dir))
if other, taken := seen[dir]; taken {
return fmt.Errorf("the components %q and %q share the directory %s: copies of one topic would collide there", other, name, dir)
}
seen[dir] = name
}
return nil
}
// exists reports whether a path is there.
func exists(name string) bool {
_, err := os.Stat(name)
return err == nil
}
+387
View File
@@ -0,0 +1,387 @@
package cli_test
import (
"os"
"path/filepath"
"strings"
"testing"
"git.vakhrushev.me/av/convy/internal/cli"
)
// readingGuide stands in for what a suite puts next to its copies: the short
// account of the language, naming every word the version line names.
const readingGuide = `# Как читать конвенцию
Слова толкуются так, и только когда написаны заглавными.
| Слово | Значение |
|---|---|
| ДОЛЖЕН | требование |
| НЕ ДОЛЖЕН | запрет |
| СЛЕДУЕТ | рекомендация |
| НЕ СЛЕДУЕТ | рекомендация против |
| ДОПУСКАЕТСЯ | разрешение |
| ПОЧЕМУ | обоснование |
| ПРИМЕРЫ | иллюстрация |
| МЕХАНИЗИРОВАНО | чем проверяется |
| СНЯТО | заглушка снятого правила |
`
// subscribable is a suite a project can take from: the fixture of the retire
// tests plus the guide that travels next to the copies.
func subscribable(t *testing.T) string {
t.Helper()
root := retirable(t)
appendRules(t, root, "conventions/lang/go/time.md", `
### GTIM-1. «Сейчас» берётся у слоя хранилища
**ДОЛЖЕН.** Текущее время приходит из `+"`store.Now()`"+`.
**ПОЧЕМУ.** Единая точка даёт гарантированный UTC и один формат.
`)
name := filepath.Join(root, "READING.md")
if err := os.WriteFile(name, []byte(readingGuide), 0o644); err != nil {
t.Fatal(err)
}
body := read(t, root, "suite.toml")
body = strings.Replace(body, `# reading = "READING.md"`, `reading = "READING.md"`, 1)
if err := os.WriteFile(filepath.Join(root, "suite.toml"), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
checkClean(t, root)
return root
}
// wired builds a project taking from that suite, and returns its root.
func wired(t *testing.T, suiteRoot string, args ...string) string {
t.Helper()
root := filepath.Join(t.TempDir(), "app")
if err := os.MkdirAll(root, 0o755); err != nil {
t.Fatal(err)
}
all := append([]string{"init", "--source", suiteRoot}, args...)
if code, out := run(t, root, "", false, all...); code != cli.OK {
t.Fatalf("convy init returned %d: %s", code, out)
}
return root
}
func TestInitAddAndPullBuildACopyOutOfTheLayers(t *testing.T) {
suiteRoot := subscribable(t)
root := wired(t, suiteRoot, "--component", "backend", "--dir", "docs/conventions", "--lang", "go")
code, out := run(t, root, "", false, "add", "time")
if code != cli.OK {
t.Fatalf("convy add returned %d: %s", code, out)
}
if !strings.Contains(out, "docs/conventions/time.md") {
t.Errorf("the assembled file is not named:\n%s", out)
}
body := read(t, root, "docs/conventions/time.md")
for _, want := range []string{
"origin: time",
"### TIME-1.",
"#### GTIM-1.",
"<!-- conv:local -->",
} {
if !strings.Contains(body, want) {
t.Errorf("the copy lacks %q:\n%s", want, body)
}
}
if strings.Contains(body, "prefix: TIME") {
t.Errorf("the front matter of a layer travelled into the copy:\n%s", body)
}
// The guide to reading a rule belongs to the suite and travels with the
// copies; README.md in the same directory belongs to the repository.
if guide := read(t, root, "docs/conventions/READING.md"); !strings.Contains(guide, "ДОПУСКАЕТСЯ") {
t.Errorf("the reading guide did not travel:\n%s", guide)
}
if manifest := read(t, root, ".conventions.toml"); !strings.Contains(manifest, `topics = ["time"]`) {
t.Errorf("the subscription was not written:\n%s", manifest)
}
if code, out := run(t, root, "", false, "check"); code != cli.OK {
t.Fatalf("checking what the tool assembled returned %d: %s", code, out)
}
}
// The whole point of the marker: what the repository wrote survives, what the
// suite wrote is replaced.
func TestPullKeepsTheLocalPartAndReplacesTheRest(t *testing.T) {
suiteRoot := subscribable(t)
root := wired(t, suiteRoot, "--component", "backend", "--dir", "docs/conventions", "--lang", "go")
run(t, root, "", false, "add", "time")
name := filepath.Join(root, "docs", "conventions", "time.md")
body := read(t, root, "docs/conventions/time.md")
body = strings.Replace(body, "**ДОЛЖЕН.** Момент времени", "**ДОЛЖЕН.** Правка выше маркера", 1)
body += "\nTIME-1 — МЕХАНИЗИРОВАНО: `internal/archrules`.\n"
if err := os.WriteFile(name, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
code, out := run(t, root, "", false, "pull")
if code != cli.OK {
t.Fatalf("convy pull returned %d: %s", code, out)
}
if !strings.Contains(out, "local part kept") {
t.Errorf("the pull did not say the local part survived:\n%s", out)
}
body = read(t, root, "docs/conventions/time.md")
if strings.Contains(body, "Правка выше маркера") {
t.Errorf("an edit above the marker survived, and it is declared not to:\n%s", body)
}
if !strings.Contains(body, "МЕХАНИЗИРОВАНО: `internal/archrules`") {
t.Errorf("the local part was lost:\n%s", body)
}
}
// A layer travels when the axis it declares agrees with the component; the base
// layer travels always.
func TestAComponentTakesOnlyTheLayersThatFitIt(t *testing.T) {
suiteRoot := subscribable(t)
root := wired(t, suiteRoot, "--component", "web", "--dir", "web/docs/conventions", "--lang", "javascript")
run(t, root, "", false, "add", "time")
body := read(t, root, "web/docs/conventions/time.md")
if !strings.Contains(body, "### TIME-1.") {
t.Errorf("the base layer did not travel:\n%s", body)
}
if strings.Contains(body, "GTIM-1") {
t.Errorf("a go layer travelled into a javascript component:\n%s", body)
}
}
func TestProjectCommandsRefuseWhatTheyCannotDo(t *testing.T) {
suiteRoot := subscribable(t)
root := wired(t, suiteRoot, "--component", "backend", "--dir", "docs/conventions", "--lang", "go")
run(t, root, "", false, "add", "time")
cases := []struct {
name string
args []string
want string
}{{
name: "a topic the suite does not declare",
args: []string{"add", "billing"},
want: "declares no topic",
}, {
name: "a topic taken twice",
args: []string{"add", "time"},
want: "subscribed to",
}, {
name: "a component that is not there",
args: []string{"pull", "--for", "mobile"},
want: "no component",
}}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
code, out := run(t, root, "", false, tc.args...)
if code == cli.OK {
t.Fatalf("the command went through:\n%s", out)
}
if !strings.Contains(out, tc.want) {
t.Errorf("the refusal does not say %q:\n%s", tc.want, out)
}
})
}
}
// A file that lost its origin key has become a document of the repository, and
// assembly has no business overwriting it.
func TestPullDoesNotOverwriteWhatIsNoLongerACopy(t *testing.T) {
suiteRoot := subscribable(t)
root := wired(t, suiteRoot, "--component", "backend", "--dir", "docs/conventions", "--lang", "go")
run(t, root, "", false, "add", "time")
name := filepath.Join(root, "docs", "conventions", "time.md")
body := read(t, root, "docs/conventions/time.md")
body = strings.Replace(body, "---\norigin: time\n---\n\n", "", 1)
if err := os.WriteFile(name, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
code, out := run(t, root, "", false, "pull")
if code == cli.OK {
t.Fatalf("the pull overwrote a document of the repository:\n%s", out)
}
if !strings.Contains(out, "no origin key") {
t.Errorf("the refusal does not say why:\n%s", out)
}
if got := read(t, root, "docs/conventions/time.md"); strings.Contains(got, "origin: time") {
t.Errorf("the file was rewritten anyway:\n%s", got)
}
}
// The context is read off the manifest lying next to you, and nothing is done
// at a guess.
func TestAProjectCommandInsideASuiteSaysWhereItIs(t *testing.T) {
suiteRoot := subscribable(t)
code, out := run(t, suiteRoot, "", false, "pull")
if code == cli.OK {
t.Fatalf("convy pull ran inside a suite:\n%s", out)
}
if !strings.Contains(out, "convy suite") {
t.Errorf("the refusal does not point at the commands of a suite:\n%s", out)
}
}
func TestListShowsWhatIsTakenAndWhatIsNot(t *testing.T) {
suiteRoot := subscribable(t)
root := wired(t, suiteRoot, "--component", "backend", "--dir", "docs/conventions", "--lang", "go")
run(t, root, "", false, "add", "time")
code, out := run(t, root, "", false, "list")
if code != cli.OK {
t.Fatalf("convy list returned %d: %s", code, out)
}
for _, want := range []string{"backend — docs/conventions", "lang=go", "time", "2 layers", "not taken", "logging"} {
if !strings.Contains(out, want) {
t.Errorf("the listing lacks %q:\n%s", want, out)
}
}
}
// The topic stands in front of the flags, and the flag package stops at the
// first argument that is not one — so "convy add time --for backend", the form
// the model writes, has to keep the component rather than drop it in silence.
func TestAddTakesTheTopicBeforeTheFlags(t *testing.T) {
suiteRoot := subscribable(t)
root := wired(t, suiteRoot, "--component", "backend", "--dir", "backend/docs", "--lang", "go")
body := read(t, root, ".conventions.toml")
body += "\n[components.web]\ndir = \"web/docs\"\ntopics = []\n"
if err := os.WriteFile(filepath.Join(root, ".conventions.toml"), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
code, out := run(t, root, "", false, "add", "logging", "--for", "web")
if code != cli.OK {
t.Fatalf("convy add returned %d: %s", code, out)
}
if !strings.Contains(out, "web/docs/logging.md") {
t.Errorf("the copy did not land in the named component:\n%s", out)
}
if manifest := read(t, root, ".conventions.toml"); !strings.Contains(manifest, `topics = ["logging"]`) {
t.Errorf("the subscription went to the wrong component:\n%s", manifest)
}
}
// Distinct directories are the only thing that tells two copies of one topic
// apart, so sharing one is an error of the manifest rather than a reason to
// rename files.
func TestTwoComponentsMayNotShareADirectory(t *testing.T) {
suiteRoot := subscribable(t)
root := wired(t, suiteRoot, "--component", "backend", "--dir", "docs/conventions", "--lang", "go")
body := read(t, root, ".conventions.toml")
body += "\n[components.web]\ndir = \"docs/conventions\"\ntopics = []\n"
if err := os.WriteFile(filepath.Join(root, ".conventions.toml"), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
code, out := run(t, root, "", false, "pull")
if code == cli.OK {
t.Fatalf("two components wrote into one directory:\n%s", out)
}
if !strings.Contains(out, "share the directory") {
t.Errorf("the refusal does not say what collides:\n%s", out)
}
}
// Bare, a project command asks; that mode is for a person, and the one with
// flags is for agents and scripts.
func TestAddAsksWhichTopicWhenToldNothing(t *testing.T) {
suiteRoot := subscribable(t)
root := wired(t, suiteRoot, "--component", "backend", "--dir", "docs/conventions", "--lang", "go")
code, out := run(t, root, "time\n", true, "add")
if code != cli.OK {
t.Fatalf("the dialogue returned %d: %s", code, out)
}
if !strings.Contains(out, "Topic") || !strings.Contains(out, "время") {
t.Errorf("the question carries no hint about what is on offer:\n%s", out)
}
if !strings.Contains(out, "docs/conventions/time.md") {
t.Errorf("the dialogue assembled nothing:\n%s", out)
}
// Without a terminal the same bare command refuses instead of blocking on
// an answer nobody is there to give.
code, out = run(t, root, "", false, "add")
if code == cli.OK {
t.Fatalf("a bare command went through with no terminal:\n%s", out)
}
if !strings.Contains(out, "no terminal") {
t.Errorf("the refusal does not say why:\n%s", out)
}
}
// The language is a level of its own, and a suite may keep its documents apart.
// Nothing else changes: the guide still travels next to the copies.
func TestTheLanguageMayLiveApartFromTheSuite(t *testing.T) {
suiteRoot := subscribable(t)
apart := filepath.Join(filepath.Dir(suiteRoot), "language")
if err := os.MkdirAll(apart, 0o755); err != nil {
t.Fatal(err)
}
if err := os.Rename(filepath.Join(suiteRoot, "READING.md"), filepath.Join(apart, "READING.md")); err != nil {
t.Fatal(err)
}
body := read(t, suiteRoot, "suite.toml")
body = strings.Replace(body, `reading = "READING.md"`, "source = \"../language\"\nreading = \"READING.md\"", 1)
if err := os.WriteFile(filepath.Join(suiteRoot, "suite.toml"), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
checkClean(t, suiteRoot)
root := wired(t, suiteRoot, "--component", "backend", "--dir", "docs/conventions", "--lang", "go")
if code, out := run(t, root, "", false, "add", "time"); code != cli.OK {
t.Fatalf("convy add returned %d: %s", code, out)
}
if guide := read(t, root, "docs/conventions/READING.md"); !strings.Contains(guide, "ДОПУСКАЕТСЯ") {
t.Errorf("the guide did not come from the level it lives on:\n%s", guide)
}
}
// The rules of the repository take a prefix on X and live below the marker. One
// standing above it would be wiped by the next pull, and saying so afterwards
// is too late.
func TestCheckCatchesALocalRuleAboveTheMarker(t *testing.T) {
suiteRoot := subscribable(t)
root := wired(t, suiteRoot, "--component", "backend", "--dir", "docs/conventions", "--lang", "go")
run(t, root, "", false, "add", "time")
name := filepath.Join(root, "docs", "conventions", "time.md")
body := read(t, root, "docs/conventions/time.md")
local := "\n### XTIM-1. Часы в тестах замораживаются\n\n" +
"**ДОЛЖЕН.** Тест берёт время у подменённого `store.Now`.\n\n" +
"**ПОЧЕМУ.** Плавающее время делает падение теста невоспроизводимым.\n"
if err := os.WriteFile(name, []byte(body+local), 0o644); err != nil {
t.Fatal(err)
}
if code, out := run(t, root, "", false, "check"); code != cli.OK {
t.Fatalf("a local rule below the marker was turned down: %d\n%s", code, out)
}
above := strings.Replace(body, "<!-- conv:local -->", local+"\n<!-- conv:local -->", 1)
if err := os.WriteFile(name, []byte(above), 0o644); err != nil {
t.Fatal(err)
}
code, out := run(t, root, "", false, "check")
if code == cli.OK {
t.Fatalf("a rule of the repository above the marker went unnoticed:\n%s", out)
}
if !strings.Contains(out, "would wipe it") {
t.Errorf("the finding does not say what is at stake:\n%s", out)
}
}
+104
View File
@@ -0,0 +1,104 @@
package cli
import (
"flag"
"fmt"
"git.vakhrushev.me/av/convy/internal/project"
)
// convy pull reassembles what the manifest lists. It never reports what
// changed: after a pull that is shown by git diff, and the decision to accept,
// to fix or to roll back is taken by a person before the commit. A second
// mechanism for comparing files, standing next to git, would answer the same
// question worse.
func runPull(env Env, args []string) ExitCode {
fs := flag.NewFlagSet("convy pull", flag.ContinueOnError)
fs.SetOutput(env.Err)
root := fs.String("root", "", "root of the project; it is looked up upwards by default")
forComponent := fs.String("for", "", "component to reassemble; every one of them by default")
if err := fs.Parse(args); err != nil {
return Usage
}
o, code := openProject(env, *root)
if code != OK {
return code
}
defer o.Close()
if err := distinctDirs(o.Manifest); err != nil {
fmt.Fprintln(env.Err, err)
return Usage
}
names, code := components(env, o.Manifest, *forComponent)
if code != OK {
return code
}
failed := 0
written := 0
for i, name := range names {
c := o.Manifest.Components[name]
if i > 0 {
fmt.Fprintln(env.Out)
}
fmt.Fprintf(env.Out, "%s → %s\n", name, c.Dir)
if c.Dir == "" {
fmt.Fprintln(env.Err, " the component names no dir, and a copy has to be written somewhere")
failed++
continue
}
if len(c.Topics) == 0 {
fmt.Fprintln(env.Out, " subscribed to nothing yet")
}
// The path column is sized to the component rather than guessed: a
// name that overruns a fixed width breaks every row below it.
width := len(c.Dir) + 1 + len(project.ReadingName)
for _, topic := range c.Topics {
width = max(width, len(c.Dir)+len(topic)+4)
}
for _, topic := range c.Topics {
made, err := project.Assemble(o.Suite, o.Root, c, topic)
if err != nil {
fmt.Fprintf(env.Err, " %s: %s\n", topic, err)
failed++
continue
}
written++
fmt.Fprintf(env.Out, " %-*s %s%s\n", width, made.Path, plural(len(made.Layers), "layer"), kept(made))
}
guide, err := project.Reading(o.LangRoot, o.Suite.Manifest.Language.Reading, o.Root, c.Dir)
if err != nil {
fmt.Fprintf(env.Err, " %s\n", err)
failed++
continue
}
fmt.Fprintf(env.Out, " %-*s the guide to reading a rule\n", width, guide)
}
fmt.Fprintf(env.Out, "\n%s assembled", plural(written, "file"))
if failed > 0 {
fmt.Fprintf(env.Out, ", %s\n", plural(failed, "failure"))
return Failed
}
fmt.Fprintln(env.Out)
fmt.Fprintln(env.Out, "git diff says what changed")
return OK
}
// kept notes that a local part was carried over, because that is the one thing
// a reassembly could have destroyed and did not.
func kept(c project.Copy) string {
switch {
case c.Created:
return ", new"
case c.Kept:
return ", local part kept"
}
return ""
}
+16 -2
View File
@@ -5,6 +5,7 @@ import (
"fmt"
"io"
"sort"
"strings"
"git.vakhrushev.me/av/convy/internal/doc"
"git.vakhrushev.me/av/convy/internal/lang"
@@ -22,7 +23,7 @@ func runSuiteList(env Env, args []string) ExitCode {
root := fs.String("root", "", "root of the suite; by default it is looked up upwards")
topic := fs.String("topic", "", "show one topic only")
langAxis := fs.String("lang", "", "language of the component, to show what it would take")
stackAxis := fs.String("stack", "", "stack of the component, to show what it would take")
stackAxis := fs.String("stack", "", "stack of the component, comma-separated, to show what it would take")
retired := fs.Bool("retired", false, "show the retired names instead of the live ones")
if err := fs.Parse(args); err != nil {
return Usage
@@ -60,7 +61,7 @@ func runSuiteList(env Env, args []string) ExitCode {
}
selecting := *langAxis != "" || *stackAxis != ""
component := suite.Component{Lang: *langAxis, Stack: *stackAxis}
component := suite.Component{Lang: split(*langAxis), Stack: split(*stackAxis)}
for i, name := range topics {
if i > 0 {
fmt.Fprintln(env.Out)
@@ -141,6 +142,19 @@ func listRetired(w io.Writer, s *suite.Suite) {
fmt.Fprintln(w, "\nnone of these names is ever handed out again")
}
// split reads a comma-separated list off the command line. An axis of a
// component is a list — a component may sit on two stacks at once — and one
// flag repeated is worse to type than one flag with commas in it.
func split(value string) []string {
var out []string
for _, part := range strings.Split(value, ",") {
if part = strings.TrimSpace(part); part != "" {
out = append(out, part)
}
}
return out
}
func sortedKeys(m map[string]string) []string {
keys := make([]string, 0, len(m))
for k := range m {
+7
View File
@@ -18,6 +18,11 @@ type Front struct {
Lang string
Stack string
Extends string
// Origin is the key a copy carries in a consuming repository: the name of
// the topic it was assembled from. It is the whole front matter of a copy
// and it appears nowhere in a suite — a file that lost it is no longer a
// copy and is never overwritten again.
Origin string
// At is the line a key was declared on, so a finding can point at the
// declaration rather than at the top of the file.
@@ -75,6 +80,8 @@ func parseFront(lines []string) (Front, int, error) {
front.Stack = value
case "extends":
front.Extends = value
case "origin":
front.Origin = value
default:
front.Unknown = append(front.Unknown, key)
}
+75
View File
@@ -292,6 +292,81 @@ func Lookup(version int, code string) (Vocabulary, error) {
return v, nil
}
// Recognize picks the vocabulary a text is written in by the words it names.
//
// A copy in a consuming repository declares its language by the version line
// and by nothing else: no manifest travels with it, and no path back to the
// suite is written anywhere in it. So a check run outside a suite learns which
// words are normative the same way a reader does — off the line the document
// carries. The version number decides between vocabularies naming the same
// words, which is what two versions of one natural language would do.
func Recognize(text string) (Vocabulary, bool) {
var best Vocabulary
for _, version := range sortedVersions() {
for _, code := range sortedCodes(version) {
v := registry[version][code]
if !namesAll(text, v.Words()) {
continue
}
if best.Version == 0 || matchesVersion(text, v.Version) {
best = v
}
}
}
return best, best.Version != 0
}
func namesAll(text string, words []string) bool {
for _, w := range words {
if !strings.Contains(text, w) {
return false
}
}
return true
}
// matchesVersion looks for the number as a whole word, so that version 1 is not
// read out of "version 12".
func matchesVersion(text string, version int) bool {
number := fmt.Sprint(version)
for i := 0; ; {
j := strings.Index(text[i:], number)
if j < 0 {
return false
}
start, end := i+j, i+j+len(number)
before := start == 0 || !isDigit(rune(text[start-1]))
after := end == len(text) || !isDigit(rune(text[end]))
if before && after {
return true
}
i = end
if i >= len(text) {
return false
}
}
}
func isDigit(r rune) bool { return r >= '0' && r <= '9' }
func sortedVersions() []int {
out := make([]int, 0, len(registry))
for v := range registry {
out = append(out, v)
}
sort.Ints(out)
return out
}
func sortedCodes(version int) []string {
out := make([]string, 0, len(registry[version]))
for c := range registry[version] {
out = append(out, c)
}
sort.Strings(out)
return out
}
// Foreign lists the words of the other vocabularies of the same version — the
// ones that give away a mixture of vocabularies. Words that coincide with the
// suite's own are dropped.
+28
View File
@@ -72,3 +72,31 @@ func TestUnknownVersionAndCode(t *testing.T) {
t.Error("an unknown vocabulary was accepted without an error")
}
}
// A copy in a consuming repository carries no manifest, so the only thing that
// says which words of it are normative is the line it names the language by.
func TestRecognizeReadsTheLanguageOffTheVersionLine(t *testing.T) {
ru, err := lang.Lookup(1, "ru")
if err != nil {
t.Fatal(err)
}
en, err := lang.Lookup(1, "en")
if err != nil {
t.Fatal(err)
}
for _, want := range []lang.Vocabulary{ru, en} {
got, ok := lang.Recognize(want.VersionLine())
if !ok {
t.Fatalf("the version line of %q was not recognized", want.Code)
}
if got.Code != want.Code || got.Version != want.Version {
t.Errorf("read as %q version %d, expected %q version %d",
got.Code, got.Version, want.Code, want.Version)
}
}
if _, ok := lang.Recognize("Обычная проза, ничего не объявляющая."); ok {
t.Errorf("prose naming no words passed for a version line")
}
}
+189
View File
@@ -0,0 +1,189 @@
package manifest
import (
"fmt"
"regexp"
"slices"
"sort"
"strings"
)
// A subscription is an array rather than a key-value pair, and the project
// manifest is edited as text for the same reason the suite one is: what the
// author wrote around the data — which component is what, why a topic is taken
// — has to survive the tool touching the file.
//
// The shape of the array survives too. An array written on one line stays on
// one line, one written down a column stays a column: reflowing it would make
// every commit that adds a topic look like a rewrite of the file.
var arrayKeyRe = regexp.MustCompile(`^(\s*)("[^"]+"|[A-Za-z0-9_-]+)\s*=\s*\[`)
// AddToList appends a value to the array under key in the given table. A table
// that has no such key gets one holding the single value.
func AddToList(source []byte, table, key, value string) ([]byte, error) {
lines := strings.Split(string(source), "\n")
start, end, ok := tableBounds(lines, table)
if !ok {
return nil, fmt.Errorf("the manifest holds no table [%s]", table)
}
from, to, found := arrayBounds(lines, start+1, end, key)
if !found {
keys, at := tableKeys(lines, start+1, end)
insert := insertionPoint(keys, at, key, lines, start, end)
entry := fmt.Sprintf(`%s = ["%s"]`, key, escape(value))
out := make([]string, 0, len(lines)+1)
out = append(out, lines[:insert]...)
out = append(out, entry)
out = append(out, lines[insert:]...)
return []byte(strings.Join(out, "\n")), nil
}
values := arrayValues(lines[from : to+1])
if slices.Contains(values, value) {
return nil, fmt.Errorf("%s already holds %q", key, value)
}
sorted := sort.StringsAreSorted(values)
values = append(values, value)
if sorted {
sort.Strings(values)
}
indent := arrayKeyRe.FindStringSubmatch(lines[from])[1]
return splice(lines, from, to, renderArray(indent, key, values, from != to)), nil
}
// RemoveFromList drops a value from the array under key.
func RemoveFromList(source []byte, table, key, value string) ([]byte, error) {
lines := strings.Split(string(source), "\n")
start, end, ok := tableBounds(lines, table)
if !ok {
return nil, fmt.Errorf("the manifest holds no table [%s]", table)
}
from, to, found := arrayBounds(lines, start+1, end, key)
if !found {
return nil, fmt.Errorf("the table [%s] holds no key %s", table, key)
}
values := arrayValues(lines[from : to+1])
i := slices.Index(values, value)
if i < 0 {
return nil, fmt.Errorf("%s does not hold %q", key, value)
}
values = slices.Delete(values, i, i+1)
indent := arrayKeyRe.FindStringSubmatch(lines[from])[1]
return splice(lines, from, to, renderArray(indent, key, values, from != to)), nil
}
// AddTable appends a table to the end of the manifest. A new component is a new
// table, and it goes last because the order of components is the author's:
// there is nothing to sort them by that would mean anything.
func AddTable(source []byte, table string, entries [][2]string) ([]byte, error) {
lines := strings.Split(string(source), "\n")
if _, _, ok := tableBounds(lines, table); ok {
return nil, fmt.Errorf("the manifest already holds the table [%s]", table)
}
for len(lines) > 0 && strings.TrimSpace(lines[len(lines)-1]) == "" {
lines = lines[:len(lines)-1]
}
block := []string{"", "[" + table + "]"}
for _, e := range entries {
block = append(block, e[0]+" = "+e[1])
}
block = append(block, "")
return []byte(strings.Join(append(lines, block...), "\n")), nil
}
// tableBounds finds the lines a table spans: its header and the line the next
// table starts on.
func tableBounds(lines []string, table string) (start, end int, ok bool) {
start = -1
for i, line := range lines {
if m := tableRe.FindStringSubmatch(line); m != nil && m[1] == table {
start = i
break
}
}
if start < 0 {
return 0, 0, false
}
end = len(lines)
for i := start + 1; i < len(lines); i++ {
if tableRe.MatchString(lines[i]) {
end = i
break
}
}
return start, end, true
}
// arrayBounds finds the first and the last line of the array under key.
func arrayBounds(lines []string, from, to int, key string) (start, end int, ok bool) {
for i := from; i < to; i++ {
m := arrayKeyRe.FindStringSubmatch(lines[i])
if m == nil || strings.Trim(m[2], `"`) != key {
continue
}
for j := i; j < to; j++ {
if strings.Contains(lines[j], "]") {
return i, j, true
}
}
return i, i, true
}
return 0, 0, false
}
var stringRe = regexp.MustCompile(`"((?:[^"\\]|\\.)*)"`)
// arrayValues pulls the strings out of an array. The array holds names — of
// topics, of languages, of stacks — and a name is a string; anything else in
// there is not a thing this tool wrote.
func arrayValues(lines []string) []string {
text := strings.Join(lines, " ")
if i := strings.Index(text, "["); i >= 0 {
text = text[i:]
}
var out []string
for _, m := range stringRe.FindAllStringSubmatch(text, -1) {
out = append(out, unescape(m[1]))
}
return out
}
// renderArray writes the array back in the shape it had.
func renderArray(indent, key string, values []string, column bool) []string {
quoted := make([]string, len(values))
for i, v := range values {
quoted[i] = `"` + escape(v) + `"`
}
if !column {
return []string{fmt.Sprintf("%s%s = [%s]", indent, key, strings.Join(quoted, ", "))}
}
out := []string{fmt.Sprintf("%s%s = [", indent, key)}
for _, q := range quoted {
out = append(out, indent+" "+q+",")
}
return append(out, indent+"]")
}
// splice replaces lines from..to inclusive with the given block.
func splice(lines []string, from, to int, block []string) []byte {
out := make([]string, 0, len(lines)+len(block))
out = append(out, lines[:from]...)
out = append(out, block...)
if to < len(lines) {
out = append(out, lines[to+1:]...)
}
return []byte(strings.Join(out, "\n"))
}
func unescape(s string) string {
s = strings.ReplaceAll(s, `\"`, `"`)
return strings.ReplaceAll(s, `\\`, `\`)
}
+98
View File
@@ -0,0 +1,98 @@
package manifest_test
import (
"strings"
"testing"
"git.vakhrushev.me/av/convy/internal/manifest"
)
const projectSource = `# What this repository takes.
source = "../conventions"
# A component is a region where every chosen layer holds at once.
[components.backend]
dir = "backend/docs/conventions"
lang = ["go"]
topics = ["errors", "time"]
[components.web]
dir = "web/docs/conventions"
topics = [
"client-logging",
]
`
func TestAddToListKeepsTheShapeOfTheArray(t *testing.T) {
out, err := manifest.AddToList([]byte(projectSource), "components.backend", "topics", "logging")
if err != nil {
t.Fatal(err)
}
body := string(out)
if !strings.Contains(body, `topics = ["errors", "logging", "time"]`) {
t.Errorf("a sorted one-line array did not stay sorted and on one line:\n%s", body)
}
if !strings.Contains(body, "# A component is a region where every chosen layer holds at once.") {
t.Errorf("the comment did not survive the edit:\n%s", body)
}
out, err = manifest.AddToList(out, "components.web", "topics", "auth")
if err != nil {
t.Fatal(err)
}
want := "topics = [\n \"auth\",\n \"client-logging\",\n]"
if !strings.Contains(string(out), want) {
t.Errorf("an array written down a column did not stay a column:\n%s", out)
}
}
func TestAddToListCreatesTheKeyItDoesNotFind(t *testing.T) {
out, err := manifest.AddToList([]byte(projectSource), "components.web", "stack", "express")
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(out), `stack = ["express"]`) {
t.Errorf("the missing key was not created:\n%s", out)
}
}
func TestAddToListRefusesWhatIsThereAlready(t *testing.T) {
_, err := manifest.AddToList([]byte(projectSource), "components.backend", "topics", "time")
if err == nil || !strings.Contains(err.Error(), "already holds") {
t.Errorf("a repeated subscription went through: %v", err)
}
if _, err := manifest.AddToList([]byte(projectSource), "components.mobile", "topics", "time"); err == nil {
t.Errorf("a table that is not there took an entry")
}
}
func TestRemoveFromList(t *testing.T) {
out, err := manifest.RemoveFromList([]byte(projectSource), "components.backend", "topics", "errors")
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(out), `topics = ["time"]`) {
t.Errorf("the value was not removed:\n%s", out)
}
if _, err := manifest.RemoveFromList([]byte(projectSource), "components.backend", "topics", "logging"); err == nil {
t.Errorf("removing what is not there went through")
}
}
func TestAddTableGoesLast(t *testing.T) {
out, err := manifest.AddTable([]byte(projectSource), "components.mobile",
[][2]string{{"dir", `"mobile/docs/conventions"`}, {"topics", "[]"}})
if err != nil {
t.Fatal(err)
}
body := string(out)
if !strings.Contains(body, "[components.mobile]\ndir = \"mobile/docs/conventions\"\ntopics = []") {
t.Errorf("the table was not written:\n%s", body)
}
if strings.Index(body, "[components.mobile]") < strings.Index(body, "[components.web]") {
t.Errorf("the new table did not go last:\n%s", body)
}
if _, err := manifest.AddTable(out, "components.mobile", nil); err == nil {
t.Errorf("a second table of the same name went through")
}
}
+16 -4
View File
@@ -27,12 +27,19 @@ const Name = "suite.toml"
// binary, and making every suite declare what is already implied buys nothing.
const DefaultLanguageCode = "ru"
// Language is the [language] section: the version of the conventions language
// and the two documents about it. The full description stays with the author of
// the suite, the short one travels into the copy.
// Language is the [language] section: the version of the conventions language,
// the natural language its words are written in, and the two documents about
// it. The full description stays with the author of the suite, the short one
// travels into the copy.
//
// Source is where those two documents live. Empty means the suite itself, which
// is where they lie while the specification of the language has no repository
// of its own; once it moves out, the same key names it without anything else
// changing — the vocabulary is picked by version and code either way.
type Language struct {
Version int `toml:"version"`
Lang string `toml:"lang"`
Source string `toml:"source"`
Description string `toml:"description"`
Reading string `toml:"reading"`
}
@@ -87,12 +94,17 @@ func Load(root string) (*Manifest, error) {
// Find walks up from start looking for a directory that holds a suite
// manifest, so that `convy suite check` works from any subdirectory of a suite.
func Find(start string) (string, error) {
return findUp(start, Name)
}
// findUp walks up from start looking for a directory holding the named file.
func findUp(start, name string) (string, error) {
dir, err := filepath.Abs(start)
if err != nil {
return "", err
}
for {
if _, err := os.Stat(filepath.Join(dir, Name)); err == nil {
if _, err := os.Stat(filepath.Join(dir, name)); err == nil {
return dir, nil
}
parent := filepath.Dir(dir)
+118
View File
@@ -0,0 +1,118 @@
package manifest
import (
"fmt"
"os"
"path/filepath"
"slices"
"sort"
"strings"
"github.com/BurntSushi/toml"
)
// ProjectName is the name of the manifest of a consuming repository. Which of
// the two manifests lies next to you tells you where you are, so the project
// one is never named like the suite one.
const ProjectName = ".conventions.toml"
// Component is one addressee of assembly inside a project: a region where every
// selected layer holds at once — one language, one set of tools, one kind of
// application (META-36).
//
// Lang and Stack are lists because a component may well take two stack layers
// at a time — sqlite and postgres in the schema topic hold together, being
// different tables of one service. Two languages never do: a line of code is
// written in one of them, which is what a component exists to separate.
type Component struct {
Dir string `toml:"dir"`
Lang []string `toml:"lang"`
Stack []string `toml:"stack"`
Topics []string `toml:"topics"`
}
// Project is a parsed .conventions.toml: where the copies are taken from and
// which components take what.
type Project struct {
// Source is the reference to the suite. How the tool reaches it — a path
// on disk, a git repository — is a matter of the reference itself.
Source string `toml:"source"`
Components map[string]Component `toml:"components"`
// Path is where the manifest was read from.
Path string `toml:"-"`
// Root is the directory the manifest lies in; every path in it is relative
// to that directory.
Root string `toml:"-"`
// Undecoded lists keys the tool does not know: a typo in a component name
// or in a key would otherwise cost a whole subscription in silence.
Undecoded []string `toml:"-"`
}
// LoadProject reads the project manifest from directory root.
func LoadProject(root string) (*Project, error) {
path := filepath.Join(root, ProjectName)
data, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("reading the project manifest: %w", err)
}
var p Project
meta, err := toml.Decode(string(data), &p)
if err != nil {
return nil, fmt.Errorf("parsing %s: %w", path, err)
}
p.Path = path
p.Root = root
for _, key := range meta.Undecoded() {
p.Undecoded = append(p.Undecoded, key.String())
}
sort.Strings(p.Undecoded)
return &p, nil
}
// FindProject walks up from start looking for a project manifest, so that a
// command works from any subdirectory of a repository.
func FindProject(start string) (string, error) {
return findUp(start, ProjectName)
}
// Names lists the components in an order stable between runs.
func (p *Project) Names() []string {
names := make([]string, 0, len(p.Components))
for name := range p.Components {
names = append(names, name)
}
sort.Strings(names)
return names
}
// Only picks the component a command works on. With a name given it is that
// one; without a name it is the single component of the project. A project of
// several components does not get one guessed for it — it gets the list.
func (p *Project) Only(name string) (string, Component, error) {
if name != "" {
c, ok := p.Components[name]
if !ok {
return "", Component{}, fmt.Errorf("the project declares no component %q; it declares: %s", name, joinNames(p.Names()))
}
return name, c, nil
}
switch len(p.Components) {
case 0:
return "", Component{}, fmt.Errorf("%s declares no component, and a copy is assembled for a component", p.Path)
case 1:
only := p.Names()[0]
return only, p.Components[only], nil
}
return "", Component{}, fmt.Errorf("the project holds several components, and the command names none: pass --for with one of %s", joinNames(p.Names()))
}
// Subscribed reports whether a component takes a topic.
func (c Component) Subscribed(topic string) bool {
return slices.Contains(c.Topics, topic)
}
func joinNames(names []string) string {
return strings.Join(names, ", ")
}
+222
View File
@@ -0,0 +1,222 @@
// Package project assembles copies of conventions inside a consuming
// repository.
//
// A copy is flat: one file per topic, the layers of the topic inside it as
// sections in the order base, language, stack. The paths of the suite are not
// reproduced — whoever checks code against a convention reads one file and does
// not gather a topic out of three places.
//
// Everything below the local marker belongs to the repository and survives
// reassembly; everything above it is rewritten. There is no three-way merge and
// no report of divergence: after a reassembly the difference is shown by git,
// and the decision is taken by a person before the commit.
package project
import (
"fmt"
"os"
"path/filepath"
"strings"
"git.vakhrushev.me/av/convy/internal/doc"
"git.vakhrushev.me/av/convy/internal/lang"
"git.vakhrushev.me/av/convy/internal/manifest"
"git.vakhrushev.me/av/convy/internal/suite"
)
// LocalMarker is the one piece of markup inside a copy that means something to
// the tool. It is single and nameless, so there is no name to be orphaned by a
// rename.
const LocalMarker = "<!-- conv:local -->"
// Copy is the outcome of assembling one topic for one component.
type Copy struct {
Topic string
// Path is where the file lies, from the root of the project.
Path string
// Layers lists the paths in the suite the file was built from.
Layers []string
// Created says the file did not exist before.
Created bool
// Kept says a local part below the marker was carried over.
Kept bool
}
// Axis is what a component asks the suite for.
func Axis(c manifest.Component) suite.Component {
return suite.Component{Lang: c.Lang, Stack: c.Stack}
}
// Assemble builds one topic for one component and writes the file. root is the
// root of the project.
func Assemble(s *suite.Suite, root string, c manifest.Component, topic string) (Copy, error) {
if !s.Manifest.TopicLive(topic) {
if s.Manifest.TopicRetired(topic) {
return Copy{}, fmt.Errorf("the suite has retired the topic %q: %s", topic, s.Manifest.Topics.Retired[topic])
}
return Copy{}, fmt.Errorf("the suite declares no live topic %q", topic)
}
layers, _ := s.Assemble(topic, Axis(c))
if len(layers) == 0 {
return Copy{}, fmt.Errorf("the topic %q has no layer this component takes", topic)
}
rel := filepath.ToSlash(filepath.Join(c.Dir, topic+".md"))
name := filepath.Join(root, filepath.FromSlash(rel))
made := Copy{Topic: topic, Path: rel}
for _, d := range layers {
made.Layers = append(made.Layers, d.Path)
}
existing, err := os.ReadFile(name)
switch {
case os.IsNotExist(err):
made.Created = true
case err != nil:
return Copy{}, err
default:
if err := ours(rel, string(existing), topic); err != nil {
return Copy{}, err
}
}
local := localPart(string(existing))
made.Kept = strings.TrimSpace(strings.TrimPrefix(local, LocalMarker)) != ""
body := Render(topic, layers, s.Vocab) + "\n\n" + local
if err := os.MkdirAll(filepath.Dir(name), 0o755); err != nil {
return Copy{}, err
}
if err := os.WriteFile(name, []byte(body), 0o644); err != nil {
return Copy{}, err
}
return made, nil
}
// ReadingName is what the reader's guide of the language is called next to the
// copies, whatever it is called where it came from.
const ReadingName = "READING.md"
// Reading puts the reader's guide of the language next to the copies. The guide
// belongs to the level above and is overwritten whole; README.md standing next
// to it belongs to the repository and is never touched.
//
// The guide travels because the copy names the language by a version and by no
// path: without the guide an agent reading a copy takes ДОПУСКАЕТСЯ for the
// everyday "you can" and loses exactly what the word was introduced for.
func Reading(langRoot, path, root, dir string) (string, error) {
if path == "" {
return "", fmt.Errorf("the suite manifest names no reading guide, and it is what travels next to the copies")
}
body, err := os.ReadFile(filepath.Join(langRoot, filepath.FromSlash(path)))
if err != nil {
return "", fmt.Errorf("reading the guide of the language: %w", err)
}
rel := filepath.ToSlash(filepath.Join(dir, ReadingName))
name := filepath.Join(root, filepath.FromSlash(rel))
if err := os.MkdirAll(filepath.Dir(name), 0o755); err != nil {
return "", err
}
if err := os.WriteFile(name, body, 0o644); err != nil {
return "", err
}
return rel, nil
}
// ours refuses to overwrite a file that is not a copy of this topic. A file
// whose origin key was taken away has stopped being a copy and become a
// document of the repository, and the tool has no business rewriting it.
func ours(rel, existing, topic string) error {
d, err := doc.Parse(rel, existing)
if err != nil {
return fmt.Errorf("%s is in the way and cannot be read: %w", rel, err)
}
switch {
case d.Front.Origin == "" && d.Front.Topic == "" && d.Front.Prefix == "":
return fmt.Errorf("%s carries no origin key: it is a document of the repository rather than a copy, and assembly would overwrite it", rel)
case d.Front.Origin == "":
return fmt.Errorf("%s carries no origin key, while it does carry the front matter of a suite file: it looks like a layer put here by hand", rel)
case d.Front.Origin != topic:
return fmt.Errorf("%s is a copy of the topic %q, and the topic %q would be assembled into the same file", rel, d.Front.Origin, topic)
}
return nil
}
// localPart returns everything from the marker down, together with the marker.
// A file that has none gets one: the marker is placed by the assembler, and a
// copy without it has nowhere to put a derogation.
func localPart(existing string) string {
lines := strings.Split(existing, "\n")
for i, line := range lines {
if strings.TrimSpace(line) != LocalMarker {
continue
}
return strings.TrimRight(strings.Join(lines[i:], "\n"), "\n") + "\n"
}
return LocalMarker + "\n"
}
// Render lays out the part of a copy that comes from the suite: the front
// matter of the copy and the layers of the topic.
func Render(topic string, layers []*doc.Document, v lang.Vocabulary) string {
parts := make([]string, 0, len(layers)+1)
parts = append(parts, "---\norigin: "+topic+"\n---")
for i, d := range layers {
parts = append(parts, layerText(d, v, i == 0))
}
return strings.Join(parts, "\n\n")
}
// layerText renders one layer.
//
// The first layer is the document: its title, its introduction, its sections.
// Every layer after it becomes a section of that document, because a layer only
// implements and narrows the base rather than standing beside it — so its own
// headings step down one level and its title becomes the heading of the
// section. The line about the language version is dropped from all but the
// first: it says the same thing three times over otherwise, and it is what
// makes the copy self-contained rather than decoration to be repeated.
func layerText(d *doc.Document, v lang.Vocabulary, first bool) string {
skip := map[int]bool{}
if !first {
if from, to, ok := versionLines(d, v); ok {
for n := from; n <= to; n++ {
skip[n] = true
}
}
}
var out []string
for n := d.Body; n <= d.Len(); n++ {
if skip[n] {
continue
}
line := d.Line(n)
if !first && !d.Fenced(n) && strings.HasPrefix(line, "#") {
line = "#" + line
}
out = append(out, line)
}
return strings.Trim(strings.Join(out, "\n"), "\n")
}
// versionLines finds the paragraph carrying the language version line: the one
// listing every key word of the vocabulary.
func versionLines(d *doc.Document, v lang.Vocabulary) (from, to int, ok bool) {
start, end := d.Preamble()
words := v.Words()
for _, p := range d.Paragraphs(start, end) {
text := p.Text()
found := true
for _, w := range words {
if !strings.Contains(text, w) {
found = false
break
}
}
if found {
return p.Start, p.End, true
}
}
return 0, 0, false
}
+133
View File
@@ -0,0 +1,133 @@
package project_test
import (
"strings"
"testing"
"git.vakhrushev.me/av/convy/internal/doc"
"git.vakhrushev.me/av/convy/internal/lang"
"git.vakhrushev.me/av/convy/internal/project"
)
const versionLine = `Ключевые слова ДОЛЖЕН, НЕ ДОЛЖЕН, СЛЕДУЕТ, НЕ СЛЕДУЕТ, ДОПУСКАЕТСЯ и метки
ПОЧЕМУ, ПРИМЕРЫ, МЕХАНИЗИРОВАНО и СНЯТО толкуются как описано в языке
конвенций версии 1 — тогда и только тогда, когда написаны заглавными.`
const baseLayer = `---
topic: time
prefix: TIME
---
# Время
Как приложение записывает моменты.
` + versionLine + `
## Правила
### TIME-1. Момент записывается в UTC
**ДОЛЖЕН.** Момент времени записывается с суффиксом Z.
**ПОЧЕМУ.** Без явного смещения не видно, в какой зоне запись сделана.
`
const goLayer = `---
topic: time
prefix: GTIM
lang: go
extends: arch/time.md
---
# Время: реализация на Go
Как базовый слой выполняется в Go-коде.
` + versionLine + `
## Правила
### GTIM-1. «Сейчас» берётся у слоя хранилища
**ДОЛЖЕН.** Текущее время приходит из ` + "`store.Now()`" + `.
**ПОЧЕМУ.** Единая точка даёт гарантированный UTC.
`
func layers(t *testing.T, bodies ...string) []*doc.Document {
t.Helper()
v, err := lang.Lookup(1, "ru")
if err != nil {
t.Fatal(err)
}
var out []*doc.Document
for i, body := range bodies {
d, err := doc.Parse("layer.md", body)
if err != nil {
t.Fatalf("layer %d: %v", i, err)
}
d.Blocks(v)
out = append(out, d)
}
return out
}
func vocab(t *testing.T) lang.Vocabulary {
t.Helper()
v, err := lang.Lookup(1, "ru")
if err != nil {
t.Fatal(err)
}
return v
}
// A copy is one document rather than three files glued together: the first
// layer is the document, and every layer after it is a section of it.
func TestRenderMakesTheLayersSectionsOfOneDocument(t *testing.T) {
body := project.Render("time", layers(t, baseLayer, goLayer), vocab(t))
for _, want := range []string{
"---\norigin: time\n---",
"\n# Время\n",
"\n## Правила\n",
"\n### TIME-1. Момент записывается в UTC\n",
"\n## Время: реализация на Go\n",
"\n### Правила\n",
"\n#### GTIM-1. «Сейчас» берётся у слоя хранилища\n",
} {
if !strings.Contains(body, want) {
t.Errorf("the copy lacks %q:\n%s", want, body)
}
}
if strings.Contains(body, "topic: time\nprefix:") {
t.Errorf("the front matter of a layer travelled into the copy:\n%s", body)
}
if strings.Contains(body, "extends:") {
t.Errorf("a path of the suite travelled into the copy:\n%s", body)
}
}
// The line about the language is what makes a copy readable on its own, and
// saying it three times over says nothing more than saying it once.
func TestRenderNamesTheLanguageOnce(t *testing.T) {
body := project.Render("time", layers(t, baseLayer, goLayer), vocab(t))
if n := strings.Count(body, "толкуются как описано"); n != 1 {
t.Errorf("the language version line appears %d times:\n%s", n, body)
}
if !strings.Contains(body, "конвенций версии 1") {
t.Errorf("the language version line is gone altogether:\n%s", body)
}
}
// A single layer is not a special case: assembly out of one layer is a copy of
// it, with no section headings invented around it.
func TestRenderOfOneLayerKeepsItsLevels(t *testing.T) {
body := project.Render("time", layers(t, baseLayer), vocab(t))
if !strings.Contains(body, "\n# Время\n") || !strings.Contains(body, "\n### TIME-1.") {
t.Errorf("a single layer was restructured:\n%s", body)
}
if strings.Contains(body, "####") {
t.Errorf("headings of a single layer were demoted:\n%s", body)
}
}
+236
View File
@@ -0,0 +1,236 @@
// Package source resolves a reference from one level of the model to the level
// above it.
//
// The levels stack up: the language describes how a rule is written, a suite
// writes its rules in that language, a project takes copies out of a suite.
// Each level is a set of files, and each lower one names the level above by a
// reference. Where that set physically lies is a question of transport rather
// than of the model — a directory on disk, a git repository, one day an HTTP
// tree or an rclone remote — so a reference names the transport first.
//
// Two transports are implemented: a path on disk and a git repository. Kind is
// an enumeration rather than a boolean because the third one is expected, and
// because a reference that cannot be resolved has to say which transport it was
// understood as before it says what went wrong.
package source
import (
"errors"
"fmt"
"os"
"os/exec"
"path/filepath"
"regexp"
"strings"
)
// Kind is the transport a reference names.
type Kind int
const (
// Local is a directory on disk.
Local Kind = iota + 1
// Git is a repository cloned to read from.
Git
)
func (k Kind) String() string {
switch k {
case Local:
return "path"
case Git:
return "git"
}
return "unknown"
}
// Ref is a parsed reference to a level.
type Ref struct {
// Raw is the reference as the manifest wrote it.
Raw string
// Kind is the transport.
Kind Kind
// Location is the path or the URL of the repository, without the revision.
Location string
// Rev is a git branch, tag or commit. Empty means the default branch of
// the repository.
Rev string
}
// String renders the reference back the way it was written.
func (r Ref) String() string {
if r.Rev == "" {
return r.Location
}
return r.Location + "#" + r.Rev
}
// scpRe catches the short form git accepts instead of a URL: user@host:path.
var scpRe = regexp.MustCompile(`^[A-Za-z0-9_.\-]+@[A-Za-z0-9_.\-]+:`)
// schemes maps a URL scheme onto the transport that serves it. Everything git
// speaks is a git reference: http and https are the two the tool is written
// against, while ssh, git and file happen to work because the clone is the same
// clone.
//
// file:// is a git reference rather than a directory on purpose. A plain path
// already says "this directory as it lies", working tree and all; file:// says
// "the same repository as it is committed", which is a different and sometimes
// wanted thing — and the difference is the reason both spellings exist.
var schemes = map[string]Kind{
"http": Git,
"https": Git,
"ssh": Git,
"git": Git,
"file": Git,
}
// Parse reads a reference. The form is "<location>" or "<location>#<revision>";
// what the location starts with decides the transport.
func Parse(raw string) (Ref, error) {
text := strings.TrimSpace(raw)
if text == "" {
return Ref{}, errors.New("the source reference is empty")
}
location, rev := text, ""
if i := strings.LastIndex(text, "#"); i >= 0 {
location, rev = strings.TrimSpace(text[:i]), strings.TrimSpace(text[i+1:])
if location == "" {
return Ref{}, fmt.Errorf("the reference %q names a revision and nothing to take it from", text)
}
if rev == "" {
return Ref{}, fmt.Errorf("the reference %q ends with # and names no revision", text)
}
}
ref := Ref{Raw: text, Location: location, Rev: rev}
scheme, _, hasScheme := strings.Cut(location, "://")
switch {
case hasScheme:
kind, known := schemes[scheme]
if !known {
return Ref{}, fmt.Errorf("the reference %q names the scheme %q, and the tool reaches a level over a path on disk or over git", text, scheme)
}
ref.Kind = kind
case scpRe.MatchString(location):
ref.Kind = Git
default:
ref.Kind = Local
}
if ref.Kind == Local {
if rev != "" {
return Ref{}, fmt.Errorf("the reference %q pins a revision of a directory on disk: a revision is a thing only a git repository has", text)
}
// A manifest is committed and travels between machines, and a path
// through a home directory means a different place on each of them. A
// relative path resolves against the manifest, which is the form that
// survives the trip.
if strings.HasPrefix(ref.Location, "~") {
return Ref{}, fmt.Errorf("the reference %q starts from a home directory, which points somewhere else on every other machine; write it relative to the manifest or in full", text)
}
}
return ref, nil
}
// Tree is a level laid out as a directory that can be read.
type Tree struct {
ref Ref
dir string
temp bool
}
// Dir is the root of the level on disk.
func (t *Tree) Dir() string { return t.dir }
// Describe rewrites a message about the tree in terms of the reference it came
// from. A fetched level lies in a temporary directory whose name says nothing
// to anyone: what the reader can act on is the reference they wrote.
func (t *Tree) Describe(err error) error {
if err == nil || t == nil {
return err
}
return errors.New(strings.ReplaceAll(err.Error(), t.dir, t.ref.String()))
}
// Close releases whatever the opening took. A directory on disk was there
// before and stays; a clone is removed.
func (t *Tree) Close() error {
if t == nil || !t.temp {
return nil
}
return os.RemoveAll(t.dir)
}
// Open makes a reference readable. A relative path resolves against base — the
// directory of the manifest that carries the reference.
//
// A git repository is cloned afresh every time, into a directory that goes away
// with the Tree. A cache would spare the second clone and buy back the question
// of what is stale in it, and the answer to "what did it look like last time"
// belongs to git in the consuming repository rather than to a cache of the tool.
func Open(r Ref, base string) (*Tree, error) {
switch r.Kind {
case Local:
dir := filepath.FromSlash(r.Location)
if !filepath.IsAbs(dir) {
dir = filepath.Join(base, dir)
}
info, err := os.Stat(dir)
if err != nil {
return nil, fmt.Errorf("the source %s: %w", r.Raw, err)
}
if !info.IsDir() {
return nil, fmt.Errorf("the source %s is a file, while a level is a directory", r.Raw)
}
return &Tree{ref: r, dir: dir}, nil
case Git:
return clone(r)
}
return nil, fmt.Errorf("the source %s names no transport the tool knows", r.Raw)
}
// clone fetches a git reference into a temporary directory.
func clone(r Ref) (*Tree, error) {
if _, err := exec.LookPath("git"); err != nil {
return nil, fmt.Errorf("the source %s is a git repository, and there is no git in PATH to fetch it with", r.Raw)
}
dir, err := os.MkdirTemp("", "convy-source-")
if err != nil {
return nil, err
}
args := []string{"clone", "--quiet", "--depth", "1"}
if r.Rev != "" {
args = append(args, "--branch", r.Rev)
}
args = append(args, r.Location, dir)
out, err := git(args...)
if err == nil {
return &Tree{ref: r, dir: dir, temp: true}, nil
}
// A commit hash is not a branch and not a tag, so --branch turns it down.
// Reaching one costs the whole history, which is why it is the second
// attempt rather than the first.
if r.Rev != "" {
if _, deep := git("clone", "--quiet", r.Location, dir); deep == nil {
if _, at := git("-C", dir, "checkout", "--quiet", r.Rev); at == nil {
return &Tree{ref: r, dir: dir, temp: true}, nil
}
}
}
os.RemoveAll(dir)
return nil, fmt.Errorf("fetching the source %s: %w\n%s", r.Raw, err, strings.TrimSpace(out))
}
func git(args ...string) (string, error) {
cmd := exec.Command("git", args...)
// A clone that stops to ask for a password would hang a command meant to
// run unattended; failing with what git said is the answer that can be
// acted on.
cmd.Env = append(os.Environ(), "GIT_TERMINAL_PROMPT=0")
out, err := cmd.CombinedOutput()
return string(out), err
}
+163
View File
@@ -0,0 +1,163 @@
package source_test
import (
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"git.vakhrushev.me/av/convy/internal/source"
)
func TestParseTellsTheTransportsApart(t *testing.T) {
cases := []struct {
raw string
kind source.Kind
location string
rev string
}{
{"../dev-conventions", source.Local, "../dev-conventions", ""},
{"/srv/conventions", source.Local, "/srv/conventions", ""},
{"https://git.example.org/av/conventions.git", source.Git, "https://git.example.org/av/conventions.git", ""},
{"http://git.example.org/av/conventions.git#v2", source.Git, "http://git.example.org/av/conventions.git", "v2"},
{"ssh://git@git.example.org:2222/av/conventions.git", source.Git, "ssh://git@git.example.org:2222/av/conventions.git", ""},
{"git@git.example.org:av/conventions.git", source.Git, "git@git.example.org:av/conventions.git", ""},
// A plain path is the directory as it lies; file:// is the same
// repository as it is committed.
{"file:///srv/conventions#main", source.Git, "file:///srv/conventions", "main"},
}
for _, tc := range cases {
t.Run(tc.raw, func(t *testing.T) {
ref, err := source.Parse(tc.raw)
if err != nil {
t.Fatalf("parsing %q: %v", tc.raw, err)
}
if ref.Kind != tc.kind {
t.Errorf("read as %s, expected %s", ref.Kind, tc.kind)
}
if ref.Location != tc.location {
t.Errorf("location %q, expected %q", ref.Location, tc.location)
}
if ref.Rev != tc.rev {
t.Errorf("revision %q, expected %q", ref.Rev, tc.rev)
}
})
}
}
func TestParseRefusesWhatItCannotMean(t *testing.T) {
cases := []struct {
name string
raw string
want string
}{
{"nothing at all", " ", "empty"},
{"a revision of a directory", "../conventions#main", "only a git repository has"},
{"a home directory", "~/projects/conventions", "every other machine"},
{"a scheme nobody serves", "rclone://remote/conventions", "over git"},
{"a hash with no revision", "https://git.example.org/c.git#", "names no revision"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
_, err := source.Parse(tc.raw)
if err == nil {
t.Fatalf("the reference %q went through", tc.raw)
}
if !strings.Contains(err.Error(), tc.want) {
t.Errorf("the refusal does not say %q: %s", tc.want, err)
}
})
}
}
func TestOpenResolvesAPathAgainstTheManifest(t *testing.T) {
base := t.TempDir()
if err := os.MkdirAll(filepath.Join(base, "vendor", "conventions"), 0o755); err != nil {
t.Fatal(err)
}
ref, err := source.Parse("vendor/conventions")
if err != nil {
t.Fatal(err)
}
tree, err := source.Open(ref, base)
if err != nil {
t.Fatalf("opening a relative path: %v", err)
}
defer tree.Close()
if tree.Dir() != filepath.Join(base, "vendor", "conventions") {
t.Errorf("resolved to %s", tree.Dir())
}
// Nothing was fetched, so nothing is released: the directory was there
// before the command and stays after it.
tree.Close()
if _, err := os.Stat(tree.Dir()); err != nil {
t.Errorf("closing removed a directory that was not fetched: %v", err)
}
}
func TestOpenRefusesAFileWhereALevelIsExpected(t *testing.T) {
base := t.TempDir()
if err := os.WriteFile(filepath.Join(base, "conventions"), []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
ref, _ := source.Parse("conventions")
if _, err := source.Open(ref, base); err == nil || !strings.Contains(err.Error(), "a level is a directory") {
t.Errorf("a file passed as a level: %v", err)
}
}
// A git reference takes what is committed rather than what lies in the working
// tree, and that is the difference between the two spellings of a local suite.
func TestOpenGitTakesTheCommittedState(t *testing.T) {
if _, err := exec.LookPath("git"); err != nil {
t.Skip("no git in PATH")
}
repo := t.TempDir()
name := filepath.Join(repo, "suite.toml")
if err := os.WriteFile(name, []byte("committed\n"), 0o644); err != nil {
t.Fatal(err)
}
for _, args := range [][]string{
{"init", "--quiet", "-b", "main"},
{"-c", "user.email=t@example.org", "-c", "user.name=t", "add", "suite.toml"},
{"-c", "user.email=t@example.org", "-c", "user.name=t", "commit", "--quiet", "-m", "first"},
} {
cmd := exec.Command("git", args...)
cmd.Dir = repo
if out, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("git %v: %v\n%s", args, err, out)
}
}
if err := os.WriteFile(name, []byte("uncommitted\n"), 0o644); err != nil {
t.Fatal(err)
}
ref, err := source.Parse("file://" + repo + "#main")
if err != nil {
t.Fatal(err)
}
tree, err := source.Open(ref, ".")
if err != nil {
t.Fatalf("cloning a local repository: %v", err)
}
body, err := os.ReadFile(filepath.Join(tree.Dir(), "suite.toml"))
if err != nil {
t.Fatal(err)
}
if strings.TrimSpace(string(body)) != "committed" {
t.Errorf("the working tree leaked into the clone: %q", body)
}
dir := tree.Dir()
if err := tree.Close(); err != nil {
t.Fatal(err)
}
if _, err := os.Stat(dir); err == nil {
t.Errorf("the clone survived the close: %s", dir)
}
}
+17 -8
View File
@@ -1,13 +1,22 @@
package suite
import "git.vakhrushev.me/av/convy/internal/doc"
import (
"slices"
// Component is what a copy is assembled for: one language, one stack, one kind
// of application. Both axes may stay empty — a flat suite has no axes at all,
// and a component of it selects the base layer and nothing else.
"git.vakhrushev.me/av/convy/internal/doc"
)
// Component is what a copy is assembled for: one language, one set of tools,
// one kind of application. Both axes may stay empty — a flat suite has no axes
// at all, and a component of it selects the base layer and nothing else.
//
// Each axis is a list because a component may take two stack layers at once:
// sqlite and postgres in the schema topic hold together, being different tables
// of one service. Two languages never hold together, and that the axis allows
// the list anyway is a matter of one shape for both rather than a licence.
type Component struct {
Lang string
Stack string
Lang []string
Stack []string
}
// Assemble picks the layers of a topic a component takes, in the order they go
@@ -34,10 +43,10 @@ func (s *Suite) Assemble(topic string, c Component) (taken, left []*doc.Document
// fits reports whether a component takes a layer.
func fits(d *doc.Document, c Component) bool {
if d.Front.Lang != "" && d.Front.Lang != c.Lang {
if d.Front.Lang != "" && !slices.Contains(c.Lang, d.Front.Lang) {
return false
}
if d.Front.Stack != "" && d.Front.Stack != c.Stack {
if d.Front.Stack != "" && !slices.Contains(c.Stack, d.Front.Stack) {
return false
}
return true