Switch to experimental Clojure grammar

[rrudakov]

Following the sogaiu/tree-sitter-clojure#65 discussion.

This PR contains a set of commit that accumulated while we were discussing changes in the grammar.

• Update queries to work with the new grammar. Queries are also improved to better skip comments and metadata nodes. Added support for some missing earlier forms (like definterface, extend-type etc).
• Better matching for docstrings for indentation and fill-paragraph function. Instead of defining custom functions we re-use docstring query now and check if node at point matches it.
• Fixed a minor bugs in clojure-ts-align + slight performance optimization by pre-compiling the query.
• Fixed a minor bug in clojure-ts-unwind.

This includes the changes from #98, so if this one is merged, I'll close the other one.

---

Before submitting a PR mark the checkboxes for the items you've done (if you
think a checkbox does not apply, then leave it unchecked):

• The commits are consistent with our contribution guidelines.
• You've added tests (if possible) to cover your change(s). Bugfix, indentation, and font-lock tests are extremely important!
• You've run M-x checkdoc and fixed any warnings in the code you've written.
• You've updated the changelog (if adding/changing user-visible functionality).
• You've updated the readme (if adding/changing user-visible functionality).

Thanks!

[rrudakov]

@bbatsov, could you please take a look?

[bbatsov]

Sure. At a glance the PR looks OK, but I'll take a deeper look tomorrow, as the changes are quite extensive.

You can close #98 and perhaps add some notes to the README that we're using a modified version of the upstream grammar and our reasoning for doing this. Probably some of this should also go in some form to the design document.

[rrudakov]

perhaps add some notes to the README that we're using a modified version of the upstream grammar and our reasoning for doing this.

This is already done, but let me know if I should extend it further.

[rrudakov]

Probably some of this should also go in some form to the design document.

Yeah... I'm still struggling to even start with that :)

[bbatsov]

This is already done, but let me know if I should extend it further.

For the README that's fine, but for the design doc I'd probably mention the summary from the end of the discussion about the creation of the experimental grammar. Overall the PR seems to be in a good shape to me, but I left small remarks here and there.

[rrudakov]

I fixed all the mentioned issues and updated the design documentation.

[sogaiu]

Not sure if it's pilot error on my end, but my Emacs 30.1 doesn't seem to have the function treesit-query-valid-p mentioned in this bit:

(defun clojure-ts--clojure-grammar-outdated-p ()
  "Return TRUE if currently installed grammar is outdated.
This function checks if `clojure-ts-mode' is compatible with the
currently installed grammar.  The simplest way to do this is to validate
a query that is valid in a previous grammar version but invalid in the
required version."
  (treesit-query-valid-p 'clojure '((sym_lit (meta_lit)))))

I think it got added to Emacs in this commit.

It looks like it got added in 2025-01 (before Emacs-30.1 was released), but may be it didn't make it in?

I don't see it via git log locally in a checkout of the emacs-30.1 tag.

---

Fairly short implementation though...

(defun treesit-query-valid-p (language query)
  "Return non-nil if QUERY is valid in LANGUAGE, nil otherwise."
  (ignore-errors
    (treesit-query-compile language query t)
    t))

[bbatsov]

Hmm, seems I don't have it as well. It's interesting that lint job didn't catch this...

[rrudakov]

Wow, that's surprising. I should have tested it on Emacs 30. Should we just copy the implementation directly to clojure-ts-mode.el?

[bbatsov]

@rrudakov I guess that would be the simplest thing for now. Hopefully it will be included in 30.2 - I heard it's right around the corner, but I'm not sure what's going to land there.

[rrudakov]

@rrudakov I guess that would be the simplest thing for now. Hopefully it will be included in 30.2 - I heard it's right around the corner, but I'm not sure what's going to land there.

I'll create a PR shortly.