Description

In a bid to reduce the chances of missing a grammar going MIA during an update as I did with the Solidity grammar mentioned at #3973 (comment), I've made a few changes to reduce unnecessary noise that can be easily rectified from the compiler without any changes to the grammars.

Changes include:

• Adding more grammar aliases for known and easily identifiable grammars, thus clearing out a few of these errors:

  Missing include in grammar: <FILENAME> (in <PATH/TO/GRAMMAR>) attempts to include <SCOPE> but the scope cannot be found
  

• Adding handling for grammars that don't use standard paths for their grammars. This is mostly for very old grammars in particular hy and conllu, both of which have regularly get missed from updates without anyone noticing 😱 because I regularly miss:

  - [ ] repository `vendor/grammars/conllu-linguist-grammar` (from https://github.com/odanoburu/conllu-linguist-grammar) (1 errors)
      - [ ] Missing scope in repository: `text.conllu` is listed in grammars.yml but cannot be found
  - [ ] repository `vendor/grammars/hy.tmLanguage` (from https://github.com/Slowki/hy.tmLanguage.git) (1 errors)
      - [ ] Missing scope in repository: `source.hy` is listed in grammars.yml but cannot be found
  

• Adding handling for known rubbish files that the compiler picks up as grammar files and can't process:

  - [ ] repository `vendor/grammars/ballerina-grammar` (from https://github.com/ballerina-platform/ballerina-grammar) (2 errors)
      - [ ] Unknown keys in grammar: `source.ballerina` (in `syntaxes/ballerina.tmLanguage`) contains invalid keys (`monarchVariables`, `tmlVariables`)
      - [ ] Grammar conversion failed. File `syntaxes/ballerina.monarch.json` failed to parse: Undeclared scope in grammar: `syntaxes/ballerina.monarch.json` has no scope name
  - [ ] repository `vendor/grammars/oz-tmbundle` (from https://github.com/eregon/oz-tmbundle) (1 errors)
      - [ ] Grammar conversion failed. File `Originals/Oz.tmLanguage` failed to parse: XML syntax error on line 23: expected element name after <
  

• Adding the ability to quickly and easily rebuild and test the grammar-compiler docker container locally with a single command by setting the REBUILD env var, eg:

  $ REBUILD=1 script/grammar-compiler compile -o linguist-grammars
  

• Adding linguist-grammars* to the ignore list so the files created by the above command that I run for each release are ignored.

So all in all, the output from script/grammar-compiler compile -o linguist-grammars has gone from 48 outstanding issues to 37 outstanding issues.

@vmg & @kivikakk the current grammar-compiler image is a little out of date and will need updating once this PR has been merged. I'm happy to do this myself, however I don't have access to push the new image. Would one of you be able to grant me access?

Your 👀 would be greatly appreciated on this PR too.

Checklist:

• I am adding new or changing current functionality
  • I have added or updated the tests for the new or changed functionality.