Skip to content

userdiff: extend Scheme support to cover other Lisp dialects #2000

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
slburson wants to merge 2 commits into
base: master
Choose a base branch
from
Open

userdiff: extend Scheme support to cover other Lisp dialects #2000

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e20ac5b
userdiff: tighten word-diff test case of the scheme driver
j6t Jan 14, 2026
fb4c8dc
userdiff: extend Scheme support to cover other Lisp dialects
slburson Jan 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Documentation/gitattributes.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -912,6 +912,7 @@ patterns are available:
- `rust` suitable for source code in the Rust language.
Copy link

@gitgitgadget gitgitgadget bot Jan 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Johannes Sixt wrote on the Git mailing list (how to reply to this email):

Am 16.01.26 um 00:18 schrieb Scott L. Burson via GitGitGadget:
> From: "Scott L. Burson" <Scott@sympoiesis.com>
> 
> Common Lisp has top-level forms, such as 'defun' and 'defmacro', that
> are not matched by the current Scheme pattern.  Also, it is more
> common in CL, when defining user macros intended as top-level forms,
> to prefix their names with "def" instead of "define"; such forms are
> also not matched.  And some top-level forms don't even begin with
> "def".
> 
> On the other hand, it is an established formatting convention in the
> Lisp community that only top-level forms start at the left margin.  So
> matching any unindented line starting with an open parenthesis is an
> acceptable heuristic; false positives will be rare.
> 
> However, there are also cases where notionally top-level forms are
> grouped together within some containing form.  At least in the Common
> Lisp community, it is conventional to indent these by two spaces, or
> sometimes one.  But matching just an open parenthesis indented by two
> spaces would be too broad; so the pattern added by this commit
> requires an indented form to start with "(def".  It is believed that
> this strikes a good balance between potential false positives and
> false negatives.

The commit message doesn't mention the changes regarding the word-diff
pattern.  I would have prefered to have them in their own patch; it
would make the patch text less obscure about what it actually changes.

> 
> Signed-off-by: Scott L. Burson <Scott@sympoiesis.com>

> diff --git a/Documentation/gitattributes.adoc b/Documentation/gitattributes.adoc
> index f20041a323..a9ce5adef9 100644
> --- a/Documentation/gitattributes.adoc
> +++ b/Documentation/gitattributes.adoc
> @@ -912,6 +912,7 @@ patterns are available:
>  - `rust` suitable for source code in the Rust language.
>  
>  - `scheme` suitable for source code in the Scheme language.
> +Also handles Emacs Lisp, Common Lisp, and most other dialects.

Saying "most dialects" immediately begs the questions "which dialects
are not covered" and "is the dialect that I'm using covered". Let's
write it this way:

- `scheme` suitable for source code in the Lisp dialects including
  Scheme, Emacs Lisp, Common Lisp.

Note the indentation of the continuation line (see the 'bash' entry, for
example).

>  
>  - `tex` suitable for source code for LaTeX documents.
>  
> diff --git a/t/t4018/scheme-lisp-defun-a b/t/t4018/scheme-lisp-defun-a
> new file mode 100644
> index 0000000000..c3c750f76d
> --- /dev/null
> +++ b/t/t4018/scheme-lisp-defun-a
> @@ -0,0 +1,4 @@
> +(defun some-func (x y z) RIGHT
> +  (let ((a x)
> +        (b y))
> +        (ChangeMe a b)))

This also demonstrates that "(let" isn't picked up. Good.

> diff --git a/t/t4018/scheme-lisp-defun-b b/t/t4018/scheme-lisp-defun-b
> new file mode 100644
> index 0000000000..21be305968
> --- /dev/null
> +++ b/t/t4018/scheme-lisp-defun-b
> @@ -0,0 +1,4 @@
> +(macrolet ((foo (x) `(bar ,x)))
> +  (defun mumble (x) ; RIGHT
> +    (when (> x 0)
> +      (foo x)))) ; ChangeMe

Indented "(defun" overrides the earlier structure that begins in the
first column. Good.

> diff --git a/t/t4018/scheme-lisp-eval-when b/t/t4018/scheme-lisp-eval-when
> new file mode 100644
> index 0000000000..5d941d7e0e
> --- /dev/null
> +++ b/t/t4018/scheme-lisp-eval-when
> @@ -0,0 +1,4 @@
> +(eval-when (:compile-toplevel :load-toplevel :execute)  ; RIGHT
> +  (set-macro-character #\?
> +		       (lambda (stream char)
> +			 `(make-pattern-variable ,(read stream)))))  ; ChangeMe

Any structure beginning in the first column is picked up. Good.

> diff --git a/t/t4018/scheme-module b/t/t4018/scheme-module-a
> similarity index 100%
> rename from t/t4018/scheme-module
> rename to t/t4018/scheme-module-a
> diff --git a/t/t4018/scheme-module-b b/t/t4018/scheme-module-b
> new file mode 100644
> index 0000000000..77bc0c5eff
> --- /dev/null
> +++ b/t/t4018/scheme-module-b
> @@ -0,0 +1,6 @@
> +(module A
> +  (export with-display-exception)
> +  (extern (display-exception display-exception))
> +  (def (with-display-exception thunk) RIGHT
> +    (with-catch (lambda (e) (display-exception e (current-error-port)) e)
> +      thunk ChangeMe)))

module-a and module-b are basically the same text. module-b changes the
last line, so that "(def" is picked up, while module-a changes the
"(extern" line, so that "(module" is picked up. Good.

> diff --git a/t/t4034/scheme/expect b/t/t4034/scheme/expect
> index 138abe9f56..72592665f1 100644
> --- a/t/t4034/scheme/expect
> +++ b/t/t4034/scheme/expect
> @@ -6,7 +6,7 @@
>  (define (<RED>myfunc a b<RESET><GREEN>my-func first second<RESET>)
>    ; This is a <RED>really<RESET><GREEN>(moderately)<RESET> cool function.
>    (<RED>this\place<RESET><GREEN>that\place<RESET> (+ 3 4))
> -  (define <RED>|the greeting|<RESET><GREEN>|a greeting|<RESET> "hello")
> +  (define <RED>|the \greeting|<RESET><GREEN>|a \greeting|<RESET> |hello there|)
>    ({<RED>}<RESET>(([<RED>]<RESET>(func-n)<RED>[<RESET>]))<RED>{<RESET>})
>    (let ((c (<RED>+ a b<RESET><GREEN>add1 first<RESET>)))
>      (format "one more than the total is %d" (<RED>add1<RESET><GREEN>+<RESET> c <GREEN>second<RESET>))))

This tests backslash between vertical bars and non-greediness of the
pattern. Good.

Using the identifier "|the \| greeting|" could make the test even more
complete, I think.

> diff --git a/t/t4034/scheme/post b/t/t4034/scheme/post
> index 0e3bab101d..450cc234f7 100644
> --- a/t/t4034/scheme/post
> +++ b/t/t4034/scheme/post
> @@ -1,7 +1,7 @@
>  (define (my-func first second)
>    ; This is a (moderately) cool function.
>    (that\place (+ 3 4))
> -  (define |a greeting| "hello")
> +  (define |a \greeting| |hello there|)
>    ({(([(func-n)]))})
>    (let ((c (add1 first)))
>      (format "one more than the total is %d" (+ c second))))
> diff --git a/t/t4034/scheme/pre b/t/t4034/scheme/pre
> index 03d77c7c43..ba8b8ac0a4 100644
> --- a/t/t4034/scheme/pre
> +++ b/t/t4034/scheme/pre
> @@ -1,7 +1,7 @@
>  (define (myfunc a b)
>    ; This is a really cool function.
>    (this\place (+ 3 4))
> -  (define |the greeting| "hello")
> +  (define |the \greeting| |hello there|)
>    ({}(([](func-n)[])){})
>    (let ((c (+ a b)))
>      (format "one more than the total is %d" (add1 c))))
> diff --git a/userdiff.c b/userdiff.c
> index fe710a68bf..b5412e6bc3 100644
> --- a/userdiff.c
> +++ b/userdiff.c
> @@ -344,14 +344,24 @@ PATTERNS("rust",
>  	 "|[0-9][0-9_a-fA-Fiosuxz]*(\\.([0-9]*[eE][+-]?)?[0-9_fF]*)?"
>  	 "|[-+*\\/<>%&^|=!:]=|<<=?|>>=?|&&|\\|\\||->|=>|\\.{2}=|\\.{3}|::"),
>  PATTERNS("scheme",
> -	 "^[\t ]*(\\(((define|def(struct|syntax|class|method|rules|record|proto|alias)?)[-*/ \t]|(library|module|struct|class)[*+ \t]).*)$",
>  	 /*
> -	  * R7RS valid identifiers include any sequence enclosed
> -	  * within vertical lines having no backslashes
> +	  * An unindented opening parenthesis identifies a top-level
> +	  * expression in all Lisp dialects.
>  	  */
> -	 "\\|([^\\\\]*)\\|"
> -	 /* All other words should be delimited by spaces or parentheses */
> -	 "|([^][)(}{[ \t])+"),
> +	 "^(\\(.*)$\n"
> +	 /* For Scheme: a possibly indented left paren followed by a keyword. */
> +	 "^[\t ]*(\\(((define|def(struct|syntax|class|method|rules|record|proto|alias)?)[-*/ \t]|(library|module|struct|class)[*+ \t]).*)$\n"
> +	 /*
> +	  * For all Lisp dialects: a slightly indented line starting with "(def".
> +	  */
> +	 "^  ?(\\([Dd][Ee][Ff].*)$",
> +	 /*
> +	  * The union of R7RS and Common Lisp symbol syntax: allows arbitrary
> +	  * strings between vertical bars, including any escaped characters.
> +	  */
> +	 "\\|([^|\\\\]|\\\\.)*\\|"
> +	 /* All other words should be delimited by spaces or parentheses. */
> +	 "|([^][)(}{ \t])+"),
>  PATTERNS("tex", "^(\\\\((sub)*section|chapter|part)\\*{0,1}\\{.*)$",
>  	 "\\\\[a-zA-Z@]+|\\\\.|([a-zA-Z0-9]|[^\x01-\x7f])+"),
>  { .name = "default", .binary = -1 },

This change matches my expectations.

-- Hannes

Copy link

@gitgitgadget gitgitgadget bot Jan 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Scott L. Burson" wrote on the Git mailing list (how to reply to this email):

On Fri, Jan 16, 2026 at 12:49 AM Johannes Sixt <j6t@kdbg.org> wrote:
>
> The commit message doesn't mention the changes regarding the word-diff
> pattern.  I would have prefered to have them in their own patch; it
> would make the patch text less obscure about what it actually changes.

Okay, will do.

> > diff --git a/Documentation/gitattributes.adoc b/Documentation/gitattributes.adoc
> > index f20041a323..a9ce5adef9 100644
> > --- a/Documentation/gitattributes.adoc
> > +++ b/Documentation/gitattributes.adoc
> > @@ -912,6 +912,7 @@ patterns are available:
> >
> >  - `scheme` suitable for source code in the Scheme language.
> > +Also handles Emacs Lisp, Common Lisp, and most other dialects.
>
> Saying "most dialects" immediately begs the questions "which dialects
> are not covered" and "is the dialect that I'm using covered". Let's
> write it this way:
>
> - `scheme` suitable for source code in the Lisp dialects including
>   Scheme, Emacs Lisp, Common Lisp.
>
> Note the indentation of the continuation line

Of course I will fix the indentation, but I don't agree with your
proposal for the text.  There are many Lisp dialects in use, indeed
probably thousands; lots of people write their own.  As previously
noted, matching an unindented open parenthesis is a very general
heuristic that is likely to work for the vast majority of dialects.
While we can't answer the question "is the dialect I'm using covered?"
for everyone, I think the text should encourage them to give the
driver a try.

So how about this:

- `scheme` suitable for source code in most Lisp dialects,
  including Scheme, Emacs Lisp, Common Lisp, and Clojure.

I've looked at some Clojure and I believe the proposed regexp will
work for it.  I think it's a good idea to mention it explicitly,
because people might search the text for it.

> Using the identifier "|the \| greeting|" could make the test even more
> complete, I think.

Agreed, will do.

-- Scott

Copy link

@gitgitgadget gitgitgadget bot Jan 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Johannes Sixt wrote on the Git mailing list (how to reply to this email):

Am 17.01.26 um 03:09 schrieb Scott L. Burson:
> So how about this:
> 
> - `scheme` suitable for source code in most Lisp dialects,
>   including Scheme, Emacs Lisp, Common Lisp, and Clojure.

OK, let's leave it at that.

-- Hannes


- `scheme` suitable for source code in the Scheme language.
Also handles Emacs Lisp, Common Lisp, and most other dialects.

- `tex` suitable for source code for LaTeX documents.

Expand Down
4 changes: 4 additions & 0 deletions t/t4018/scheme-lisp-defun-a
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
(defun some-func (x y z) RIGHT
(let ((a x)
(b y))
(ChangeMe a b)))
4 changes: 4 additions & 0 deletions t/t4018/scheme-lisp-defun-b
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
(macrolet ((foo (x) `(bar ,x)))
(defun mumble (x) ; RIGHT
(when (> x 0)
(foo x)))) ; ChangeMe
4 changes: 4 additions & 0 deletions t/t4018/scheme-lisp-eval-when
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
(eval-when (:compile-toplevel :load-toplevel :execute) ; RIGHT
(set-macro-character #\?
(lambda (stream char)
`(make-pattern-variable ,(read stream))))) ; ChangeMe
0 t/t4018/scheme-module → t/t4018/scheme-module-a
View file
Open in desktop
File renamed without changes.
6 changes: 6 additions & 0 deletions t/t4018/scheme-module-b
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
(module A
(export with-display-exception)
(extern (display-exception display-exception))
(def (with-display-exception thunk) RIGHT
(with-catch (lambda (e) (display-exception e (current-error-port)) e)
thunk ChangeMe)))
5 changes: 3 additions & 2 deletions t/t4034/scheme/expect
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,11 @@
<BOLD>index 74b6605..63b6ac4 100644<RESET>
<BOLD>--- a/pre<RESET>
<BOLD>+++ b/post<RESET>
<CYAN>@@ -1,6 +1,6 @@<RESET>
<CYAN>@@ -1,7 +1,7 @@<RESET>
(define (<RED>myfunc a b<RESET><GREEN>my-func first second<RESET>)
; This is a <RED>really<RESET><GREEN>(moderately)<RESET> cool function.
(<RED>this\place<RESET><GREEN>that\place<RESET> (+ 3 4))
(define <RED>some-text<RESET><GREEN>|a greeting|<RESET> "hello")
(define <RED>|the \greeting|<RESET><GREEN>|a \greeting|<RESET> |hello there|)
({<RED>}<RESET>(([<RED>]<RESET>(func-n)<RED>[<RESET>]))<RED>{<RESET>})
(let ((c (<RED>+ a b<RESET><GREEN>add1 first<RESET>)))
(format "one more than the total is %d" (<RED>add1<RESET><GREEN>+<RESET> c <GREEN>second<RESET>))))
3 changes: 2 additions & 1 deletion t/t4034/scheme/post
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
(define (my-func first second)
; This is a (moderately) cool function.
(that\place (+ 3 4))
(define |a greeting| "hello")
(define |a \greeting| |hello there|)
({(([(func-n)]))})
(let ((c (add1 first)))
(format "one more than the total is %d" (+ c second))))
3 changes: 2 additions & 1 deletion t/t4034/scheme/pre
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
(define (myfunc a b)
; This is a really cool function.
(this\place (+ 3 4))
(define some-text "hello")
(define |the \greeting| |hello there|)
({}(([](func-n)[])){})
(let ((c (+ a b)))
(format "one more than the total is %d" (add1 c))))
22 changes: 16 additions & 6 deletions userdiff.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -344,14 +344,24 @@ PATTERNS("rust",
"|[0-9][0-9_a-fA-Fiosuxz]*(\\.([0-9]*[eE][+-]?)?[0-9_fF]*)?"
"|[-+*\\/<>%&^|=!:]=|<<=?|>>=?|&&|\\|\\||->|=>|\\.{2}=|\\.{3}|::"),
PATTERNS("scheme",
"^[\t ]*(\\(((define|def(struct|syntax|class|method|rules|record|proto|alias)?)[-*/ \t]|(library|module|struct|class)[*+ \t]).*)$",
/*
* R7RS valid identifiers include any sequence enclosed
* within vertical lines having no backslashes
* An unindented opening parenthesis identifies a top-level
* expression in all Lisp dialects.
*/
"\\|([^\\\\]*)\\|"
/* All other words should be delimited by spaces or parentheses */
"|([^][)(}{[ \t])+"),
"^(\\(.*)$\n"
/* For Scheme: a possibly indented left paren followed by a keyword. */
"^[\t ]*(\\(((define|def(struct|syntax|class|method|rules|record|proto|alias)?)[-*/ \t]|(library|module|struct|class)[*+ \t]).*)$\n"
/*
* For all Lisp dialects: a slightly indented line starting with "(def".
*/
"^ ?(\\([Dd][Ee][Ff].*)$",
/*
* The union of R7RS and Common Lisp symbol syntax: allows arbitrary
* strings between vertical bars, including any escaped characters.
*/
"\\|([^|\\\\]|\\\\.)*\\|"
/* All other words should be delimited by spaces or parentheses. */
"|([^][)(}{ \t])+"),
PATTERNS("tex", "^(\\\\((sub)*section|chapter|part)\\*{0,1}\\{.*)$",
"\\\\[a-zA-Z@]+|\\\\.|([a-zA-Z0-9]|[^\x01-\x7f])+"),
{ .name = "default", .binary = -1 },
Expand Down
Loading