diff --git a/CHANGELOG.md b/CHANGELOG.md index 5eef351..34efce0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,7 @@ this project aims to follow [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- feat(res-to-affine): Phase 3 slice 2 — `--translate` now also handles record types (→ `struct`) and generics (type params `'a` → `[A]`) across aliases / sums / records; `mutable`/optional-`?` records, qualified paths, and nested generics are still skipped (never guessed); every emitted form verified compilable via `main.exe check` (Refs #57) - feat(res-to-affine): Phase 3 slice 1 — `--translate` renders fully-structural type declarations (primitive aliases + simple sum types) into compilable AffineScript; conservative (generics / qualified paths / records / non-primitive payloads are skipped, never guessed); walker-only (Refs #57) - feat(stdlib/Http): RSR rewire — surface `hpm-http-rsr` Zig FFI (10 server-side externs: listen / port / free / accept / method / path / header / body / respond / request-free) + opaque `HpmHttpServer` + `HpmHttpRequest` types; native-only (#425) - feat(stdlib/json): v0.3 — RSR rewire to `hpm-json-rsr` Zig FFI (11 externs + opaque `HpmJsonValue` + `parse` / `to_json`), Deno-ESM lowering via `__as_hpmJson*` shims (#421) diff --git a/tools/res-to-affine/README.md b/tools/res-to-affine/README.md index 42bb744..2907bfe 100644 --- a/tools/res-to-affine/README.md +++ b/tools/res-to-affine/README.md @@ -151,31 +151,42 @@ where re-decomposition is genuinely required. Phase 3 is when the tool earns its keep on idaptik's 542 files. -**Phase 3 slice 1 (`--translate`, landed).** The first translation slice -renders the two most mechanical, **module-qualified-path-independent** -shapes into compilable AffineScript: - -| ReScript | AffineScript | -|---|---| -| `type userId = int` | `type UserId = Int` | -| `type color = Red \| Green \| Blue` | `type Color =`
` \| Red`
` \| Green`
` \| Blue` | -| `type shape = Circle(float) \| Rect(int, int)` | `type Shape =`
` \| Circle(Float)`
` \| Rect(Int, Int)` | - -It is **conservative by construction**: a declaration that uses type -parameters/generics, a qualified path (`Belt.Map.t`), a record body, -non-primitive references, a GADT return annotation, or a variant spread -is *skipped* — it stays in the marker block + quoted original, never -mis-translated. Lower-case ReScript type names are capitalised so they -are referenceable AffineScript type constructors (`lib/parser.mly` reads -a lower-case name in type position as a type *variable*, not a -constructor). Translation is walker-only (it needs the AST); with -`--engine=scanner` the flag is a no-op. - -Deliberately **deferred to later Phase-3 slices**: record types, generic -type declarations, module-qualified references (now that the -[#228](https://github.com/hyperpolymath/affinescript/issues/228) grammar -gap is closed), `let`-to-`const` for literal bindings, and the -`switch`→`match` expression rewrite (which needs body translation). +**Phase 3 (`--translate`, landed).** The translation path renders the +fully-structural type declarations into compilable AffineScript. Every +generated form below is verified by the compiler itself (`main.exe check` +→ *Type checking passed*). + +| ReScript | AffineScript | Slice | +|---|---|---| +| `type userId = int` | `type UserId = Int` | 1 | +| `type color = Red \| Green \| Blue` | `type Color =`
` \| Red`
` \| Green`
` \| Blue` | 1 | +| `type shape = Circle(float) \| Rect(int, int)` | `type Shape =`
` \| Circle(Float)`
` \| Rect(Int, Int)` | 1 | +| `type point = {x: int, y: int}` | `struct Point {`
` x: Int,`
` y: Int`
`}` | 2 | +| `type box<'a> = {value: 'a}` | `struct Box[A] {`
` value: A`
`}` | 2 | +| `type option<'a> = None \| Some('a)` | `type Option[A] =`
` \| None`
` \| Some(A)` | 2 | +| `type id<'a> = 'a` | `type Id[A] = A` | 2 | + +It is **conservative by construction**: a declaration is translated only +when every part is representable — a qualified-path reference +(`Belt.Map.t`), a non-primitive/opaque reference, a nested generic +(`array`), a GADT return, a variant spread, an object type, or a +record with a `mutable` or optional-`?` field causes the whole decl to be +*skipped* (it stays in the marker block + quoted original, never +mis-translated). Two normalisations make the output referenceable: +lower-case ReScript type names are capitalised (`color` → `Color`) and +type variables are mapped (`'a` → `A`), because `lib/parser.mly` reads a +lower-case name in type position as a type *variable*, not a constructor. +Translation is walker-only (it needs the AST); with `--engine=scanner` +the flag is a no-op. + +Deliberately **deferred to later Phase-3 slices**: `let`-to-`const` for +literal bindings, the `switch`→`match` expression rewrite (needs body +translation), and **module-qualified references** — these now *parse* +(the [#228](https://github.com/hyperpolymath/affinescript/issues/228) +grammar gap closed), but a faithful `Belt.Map.t` → `Belt::Map::T` would +not *resolve* against a target module that doesn't exist yet, so emitting +it would break the "every translated form type-checks" guarantee. It +waits for a module-mapping story. ## Corpus run @@ -203,8 +214,9 @@ The fixture under `test/fixtures/sample.res` is synthetic and exercises every Phase-1 anti-pattern; `test/fixtures/phase2c.res` exercises the two anti-patterns that are walker-only by construction (`inline-callback-record`, `oversized-function`); `test/fixtures/phase3.res` -exercises the Phase-3 `--translate` slice (structural type-declaration -translation, plus the generic/qualified/non-type forms it must skip). +and `test/fixtures/phase3b.res` exercise the Phase-3 `--translate` path +(aliases / sums / generics / records → compilable AffineScript, plus the +qualified / mutable / optional / non-type forms it must skip). Real `.res` files from the estate (e.g. `gitbot-fleet/bots/sustainabot/bot-integration/ src/*.res`) can be run ad hoc through the CLI without changes to the diff --git a/tools/res-to-affine/test/fixtures/phase3.res b/tools/res-to-affine/test/fixtures/phase3.res index c117fc5..712ac26 100644 --- a/tools/res-to-affine/test/fixtures/phase3.res +++ b/tools/res-to-affine/test/fixtures/phase3.res @@ -6,10 +6,10 @@ // type userId = int -> type UserId = Int // type color = Red | ... -> type Color = | Red | Green | Blue // type shape = Circle(...) -> type Shape = | Circle(Float) | Rect(Int, Int) -// The trailing `let`/`switch` is NOT a type declaration and must remain a -// TODO island (absent from the translation list). The generic and -// qualified type decls below must also be skipped (slice 1 is monomorphic -// and unqualified). +// The trailing `let`/`switch` is NOT a type declaration and stays a TODO +// island (absent from the translation list). The generic `box` below now +// translates too (slice 2: type parameters); the qualified-path decl stays +// skipped (qualified-path RHS is deferred — it would parse but not resolve). type userId = int @@ -22,10 +22,10 @@ type shape = | Circle(float) | Rect(int, int) -// Skipped in slice 1: type parameters (generic). +// Slice 2: type parameters now translate -> type Box[A] = | Box(A) type box<'a> = Box('a) -// Skipped in slice 1: qualified-path RHS. +// Skipped: qualified-path RHS (deferred). type theirMap = Belt.Map.t // Not a type declaration: stays a TODO island. diff --git a/tools/res-to-affine/test/fixtures/phase3b.res b/tools/res-to-affine/test/fixtures/phase3b.res new file mode 100644 index 0000000..e2cbe84 --- /dev/null +++ b/tools/res-to-affine/test/fixtures/phase3b.res @@ -0,0 +1,28 @@ +// SPDX-License-Identifier: MIT +// Phase 3 slice 2 fixture: record types (-> struct) and generics. +// type point = {x: int, y: int} -> struct Point { x: Int, y: Int } +// type box<'a> = {value: 'a} -> struct Box[A] { value: A } +// type id<'a> = 'a -> type Id[A] = A +// Records with `mutable` or optional `?` fields are SKIPPED (their +// semantics can't be dropped). + +type point = { + x: int, + y: int, +} + +type box<'a> = { + value: 'a, +} + +type id<'a> = 'a + +// SKIPPED: mutable field — AffineScript struct fields can't carry it. +type counter = { + mutable count: int, +} + +// SKIPPED: optional field — `?` has no struct equivalent. +type config = { + verbose?: bool, +} diff --git a/tools/res-to-affine/test/test_walker.ml b/tools/res-to-affine/test/test_walker.ml index 3b1653e..4a6ff15 100644 --- a/tools/res-to-affine/test/test_walker.ml +++ b/tools/res-to-affine/test/test_walker.ml @@ -225,9 +225,18 @@ let translate_phase3_blob () = let test_translate_count () = skip_unless_ready (); + (* userId, color, shape, and (slice 2) the generic box — 4. theirMap + (qualified) and the let/switch stay skipped. *) Alcotest.(check int) - "exactly the three structural type decls are translated" - 3 (List.length (translate_phase3 ())) + "four structural type decls are translated" + 4 (List.length (translate_phase3 ())) + +let test_translate_generic_sum () = + skip_unless_ready (); + let blob = translate_phase3_blob () in + Alcotest.(check bool) + "generic sum -> type Box[A] = | Box(A)" + true (contains blob "type Box[A] =" && contains blob "| Box(A)") let test_translate_alias () = skip_unless_ready (); @@ -256,15 +265,69 @@ let test_translate_payload_sum () = let test_translate_skips_non_structural () = skip_unless_ready (); let blob = translate_phase3_blob () in - (* the generic Box, qualified Belt.Map.t, and the let/switch must all be - absent from the translation — slice 1 never guesses them. *) + (* the qualified Belt.Map.t and the let/switch must stay absent — the tool + never guesses them; and no raw ReScript type-var ['a] leaks through. *) let leaked = - contains blob "switch" || contains blob "area" || contains blob "Box" + contains blob "switch" || contains blob "area" || contains blob "Belt" || contains blob "'a" in - Alcotest.(check bool) "non-structural / generic / qualified forms skipped" + Alcotest.(check bool) "qualified / non-type forms skipped, no raw type-var" false leaked +(* ---- Phase 3 slice 2: records (-> struct) and generics -------------------- + + [fixtures/phase3b.res] holds a record (-> struct), a generic record + (-> struct with type params), a generic alias, and two records that must + be skipped (mutable + optional fields). *) + +let phase3b_fixture = "fixtures/phase3b.res" + +let translate_phase3b () = + let source = read_file phase3b_fixture in + let path = Filename.concat (Sys.getcwd ()) phase3b_fixture in + Walker.translate ~grammar_dir:(grammar_dir ()) ~path ~source + +let translate_phase3b_blob () = + String.concat "\n" (List.map snd (translate_phase3b ())) + +let test_translate_b_count () = + skip_unless_ready (); + (* point, box, id translate; counter (mutable) and config (optional) skip. *) + Alcotest.(check int) + "three of five record/generic decls translate" + 3 (List.length (translate_phase3b ())) + +let test_translate_record () = + skip_unless_ready (); + let blob = translate_phase3b_blob () in + let ok = + contains blob "struct Point {" && contains blob "x: Int" + && contains blob "y: Int" + in + Alcotest.(check bool) "record -> struct with mapped field types" true ok + +let test_translate_generic_record () = + skip_unless_ready (); + let blob = translate_phase3b_blob () in + let ok = contains blob "struct Box[A] {" && contains blob "value: A" in + Alcotest.(check bool) "generic record -> struct with type params" true ok + +let test_translate_generic_alias () = + skip_unless_ready (); + Alcotest.(check bool) + "generic alias -> type Id[A] = A" + true (contains (translate_phase3b_blob ()) "type Id[A] = A") + +let test_translate_b_skips () = + skip_unless_ready (); + let blob = translate_phase3b_blob () in + (* mutable + optional records must be skipped, never silently flattened. *) + let leaked = + contains blob "mutable" || contains blob "count" + || contains blob "verbose" || contains blob "?" + in + Alcotest.(check bool) "mutable / optional records skipped" false leaked + let () = Alcotest.run "res-to-affine-walker" [ @@ -294,7 +357,7 @@ let () = ] ); ( "walker-phase3-translate", [ - Alcotest.test_case "three structural type decls translated" + Alcotest.test_case "four structural type decls translated" `Quick test_translate_count; Alcotest.test_case "primitive alias -> type UserId = Int" `Quick test_translate_alias; @@ -302,7 +365,22 @@ let () = `Quick test_translate_nullary_sum; Alcotest.test_case "primitive-payload sum -> mapped params" `Quick test_translate_payload_sum; - Alcotest.test_case "generic / qualified / non-type forms skipped" + Alcotest.test_case "generic sum -> type Box[A] = | Box(A)" + `Quick test_translate_generic_sum; + Alcotest.test_case "qualified / non-type forms skipped" `Quick test_translate_skips_non_structural; ] ); + ( "walker-phase3b-records-generics", + [ + Alcotest.test_case "three of five record/generic decls translate" + `Quick test_translate_b_count; + Alcotest.test_case "record -> struct" + `Quick test_translate_record; + Alcotest.test_case "generic record -> struct[A]" + `Quick test_translate_generic_record; + Alcotest.test_case "generic alias -> type Id[A] = A" + `Quick test_translate_generic_alias; + Alcotest.test_case "mutable / optional records skipped" + `Quick test_translate_b_skips; + ] ); ] diff --git a/tools/res-to-affine/walker.ml b/tools/res-to-affine/walker.ml index 84bc35d..bd2437b 100644 --- a/tools/res-to-affine/walker.ml +++ b/tools/res-to-affine/walker.ml @@ -641,16 +641,22 @@ let child_with_field name n = let has_child_type ty n = List.exists (fun c -> c.ntype = ty) n.children -(* A type node usable as an alias RHS or a variant payload. Slice 1 only - knows primitive type_identifiers; anything else yields [None]. *) -let translate_prim_type ~source n = - if n.ntype = "type_identifier" - then rescript_prim_to_affine (node_text ~source n) - else None - -(* Render one variant_declaration, or [None] if it carries a non-primitive +(* A type node usable as an alias RHS, a variant payload, or a record-field + type. Translates primitive type_identifiers and in-scope type parameters + ([params] maps a ReScript type-var like ['a] to its AffineScript name like + [A]); anything else — generics, qualified paths, tuples, nested records — + yields [None], so the whole declaration is skipped. *) +let translate_type_ref ~params ~source n = + if n.ntype <> "type_identifier" then None + else + let t = node_text ~source n in + match List.assoc_opt t params with + | Some affine -> Some affine + | None -> rescript_prim_to_affine t + +(* Render one variant_declaration, or [None] if it carries a non-translatable payload or a GADT return annotation. *) -let translate_variant ~source vd = +let translate_variant ~params ~source vd = if has_child_type "type_annotation" vd then None else match @@ -667,7 +673,7 @@ let translate_variant ~source vd = let rec go acc = function | [] -> Some (List.rev acc) | t :: rest -> - (match translate_prim_type ~source t with + (match translate_type_ref ~params ~source t with | Some s -> go (s :: acc) rest | None -> None) in @@ -676,56 +682,149 @@ let translate_variant ~source vd = | Some ss -> Some (Printf.sprintf "%s(%s)" cname (String.concat ", " ss)))) +(* AffineScript type-parameter name for a ReScript one: strip the leading + apostrophe and capitalise, so ['a] -> [A] (a referenceable TyCon in the + body). [None] if the result isn't a usable upper_ident. *) +let affine_type_param rescript_tp = + let s = + if String.length rescript_tp > 0 && rescript_tp.[0] = '\'' + then String.sub rescript_tp 1 (String.length rescript_tp - 1) + else rescript_tp + in + to_type_con s + +(* Extract the decl's type parameters as a [(rescript_name, affine_name)] map + plus the ["[A, B]"] suffix to print after the type name. [None] if a + parameter can't be represented (whole decl skipped). With no type + parameters, returns [Some ([], "")]. *) +let extract_type_params ~source tb = + match List.find_opt (fun c -> c.ntype = "type_parameters") tb.children with + | None -> Some ([], "") + | Some tps -> + let rescript_names = + List.filter_map + (fun c -> + if c.ntype = "type_identifier" then Some (node_text ~source c) + else None) + tps.children + in + let rec go assoc affines = function + | [] -> Some (List.rev assoc, List.rev affines) + | rs :: rest -> + (match affine_type_param rs with + | Some aff -> go ((rs, aff) :: assoc) (aff :: affines) rest + | None -> None) + in + (match go [] [] rescript_names with + | None | Some (_, []) -> None + | Some (assoc, affines) -> + Some (assoc, "[" ^ String.concat ", " affines ^ "]")) + +(* Render a record_type as AffineScript struct fields, or [None]. ReScript + [mutable] and optional-[?] fields are anonymous tokens (absent from the + parse tree), so we detect them in the field's source text and refuse the + whole record rather than silently drop the semantics. *) +let translate_record_fields ~params ~source rt = + let members = rt.children in + if members = [] then None + (* anything that isn't a plain record field (spreads, object members) *) + else if List.exists (fun c -> c.ntype <> "record_type_field") members then None + else + let rec go acc = function + | [] -> Some (List.rev acc) + | f :: rest -> + let ftext = String.trim (node_text ~source f) in + if starts_with "mutable" ftext || String.contains ftext '?' then None + else + (match + ( List.find_opt + (fun c -> c.ntype = "property_identifier") f.children, + List.find_opt (fun c -> c.ntype = "type_annotation") f.children ) + with + | Some name_node, Some ann -> + (match ann.children with + | ty :: _ -> + (match translate_type_ref ~params ~source ty with + | Some t -> + go + ((node_text ~source name_node ^ ": " ^ t) :: acc) + rest + | None -> None) + | [] -> None) + | _ -> None) + in + go [] members + (* Translate one type_binding into an AffineScript declaration, or [None]. *) let translate_type_binding ~source tb = - if has_child_type "type_parameters" tb then None - else - match child_with_field "name" tb with - | None -> None - | Some nn when nn.ntype <> "type_identifier" -> None (* qualified path *) - | Some nn -> - (match to_type_con (node_text ~source nn) with - | None -> None - | Some tycon -> - (match - List.find_opt (fun c -> c.ntype = "variant_type") tb.children - with - | Some vt -> - if has_child_type "variant_type_spread" vt then None - else - let vds = - List.filter - (fun c -> c.ntype = "variant_declaration") vt.children - in - let rec go acc = function - | [] -> Some (List.rev acc) - | vd :: rest -> - (match translate_variant ~source vd with - | Some s -> go (s :: acc) rest - | None -> None) - in - (match go [] vds with - | None | Some [] -> None - | Some arms -> - let body = - String.concat "\n" - (List.map (fun a -> " | " ^ a) arms) - in - Some (Printf.sprintf "type %s =\n%s" tycon body)) - | None -> - (* alias: the non-name RHS type node *) - (match - List.find_opt - (fun c -> - c.field <> Some "name" && c.ntype = "type_identifier") - tb.children - with - | None -> None - | Some r -> - (match translate_prim_type ~source r with - | None -> None - | Some prim -> - Some (Printf.sprintf "type %s = %s" tycon prim))))) + match child_with_field "name" tb with + | None -> None + | Some nn when nn.ntype <> "type_identifier" -> None (* qualified name *) + | Some nn -> + (match to_type_con (node_text ~source nn) with + | None -> None + | Some tycon -> + (match extract_type_params ~source tb with + | None -> None + | Some (params, suffix) -> + let header = tycon ^ suffix in + (match + List.find_opt (fun c -> c.ntype = "variant_type") tb.children + with + | Some vt -> + if has_child_type "variant_type_spread" vt then None + else + let vds = + List.filter + (fun c -> c.ntype = "variant_declaration") + vt.children + in + let rec go acc = function + | [] -> Some (List.rev acc) + | vd :: rest -> + (match translate_variant ~params ~source vd with + | Some s -> go (s :: acc) rest + | None -> None) + in + (match go [] vds with + | None | Some [] -> None + | Some arms -> + let body = + String.concat "\n" + (List.map (fun a -> " | " ^ a) arms) + in + Some (Printf.sprintf "type %s =\n%s" header body)) + | None -> + (match + List.find_opt + (fun c -> c.ntype = "record_type") tb.children + with + | Some rt -> + (match translate_record_fields ~params ~source rt with + | None -> None + | Some fields -> + let body = + String.concat ",\n" + (List.map (fun f -> " " ^ f) fields) + in + Some + (Printf.sprintf "struct %s {\n%s\n}" header body)) + | None -> + (* alias: the non-name RHS type node *) + (match + List.find_opt + (fun c -> + c.field <> Some "name" + && c.ntype = "type_identifier") + tb.children + with + | None -> None + | Some r -> + (match translate_type_ref ~params ~source r with + | None -> None + | Some rhs -> + Some + (Printf.sprintf "type %s = %s" header rhs))))))) (* Walk the tree; translate every module-top-level type_declaration's bindings. Returns [(source_line, affinescript)] in tree order. *)