New version of the CodeStripper that was previously used, which can be found at https://github.com/sebivenlo/codestripper.
The reason for the switch is to not be dependent on Ant as a build system and the possibility to easily add more tags.
| Command | Tag(s) | Description |
|---|---|---|
| Add | cs:add:<text> |
Add the text (without the tag in front) |
| Ignore | cs:ignore |
Ignore the entire file, only valid on the first line |
| Remove line | cs:remove |
Remove the line |
| Remove range | cs:remove:start/cs:remove:end |
Remove all text in between tags |
| Replace | cs:replace:<replacement> |
Replace the text in front by the replacement, keeps whitespace |
| Uncomment | cs:uncomment:start/cs:uncomment:end |
Uncomment all lines in between tags |
To support the old CodeStripper, the legacy tag Start Solution::replacewith::/End Solution::replacewith:: is still supported for now. This tag does both the Remove and Replace in one go.
CodeStripper can be used as a Python Module and as a command line tool. The command line tool has the following interface.
| Flag | Long form | Description | Default value | Required |
|---|---|---|---|---|
<positional> |
None | files to include for code stripping (glob) | None | True |
| -x | --exclude | files to exclude for code stripping (glob), can be repeated | None | False |
| -c | --comment | comment symbol(s) for a file extension, format <extension>:<open> or <extension>:<open>:<close> (e.g. .java://), the extension starts with a . and is case-insensitive, can be repeated |
built-in mapping (see below) | False |
| -o | --output | the output directory to store the stripped files | out | False |
| -r | --recursive | do NOT use recursive globs for include/exclude | True | False |
| -v | --verbosity | increase output verbosity (can be repeated) | 0 | False |
| -d | --dry-run | execute a dry run, prints the stripped files (with a ==> <file> <== header) to stdout instead of writing them |
False | False |
| -w | --working-directory | set the working directory for include/exclude | pwd | False |
| -e | --fail-on-error | do NOT fail if an error occurs during code stripping | fail on error | False |
| -b | --binary | what to do if a binary file is matched: fail, ignore or include |
fail | False |
| -u | --unknown | what to do if a file with an unknown extension is matched: fail, ignore or include |
fail | False |
The comment symbol is chosen based on the file extension. The following extensions are supported by default:
| Comment | Extensions |
|---|---|
// |
.java, .cs, .js, .php, .swift |
# |
.py, .r, .ps1, .rb, .yml, .yaml, files without extension |
% |
.tex, .m |
-- |
.sql, .lua |
<!-- --> |
.xml |
(* *) |
.ml |
Use -c to add or override an extension, for example -c .kt:// or -c .html:'<!--':'-->'.
from codestripper.code_stripper import strip_files
stripped = strip_files(["src/Test.java"], working_directory=".", output="out", dry_run=False)strip_files returns the files that were stripped. Files that fail (e.g. an invalid tag) are logged and the other files are still processed, after which a StripError is raised. Pass fail_on_error=False to only log the errors, like the -e flag of the command line tool.
This section contains examples for all supported tags.
Input:
public class Test {
//cs:add:private final String test = "test";
}Output:
public class Test {
private final String test = "test";
}Input:
//cs:ignore
public class Test {
private final String test = "test";
}Output: No output, file is ignored
Input:
public class Test {
private final String test = "test";//cs:remove
}Output:
public class Test {
}Input:
public class Test {
//cs:remove:start
private final String test = "test";
private final int count = 0;
//cs:remove:end
private final boolean keep = true;
}Output:
public class Test {
private final boolean keep = true;
}Input:
public class Test {
private final boolean keep = false;//cs:replace://TODO: add fields
}Output:
public class Test {
//TODO: add fields
}Input:
public class Test {
//cs:uncomment:start
//private final String example = "example";
//private final boolean isTestCode = true;
//cs:uncomment:end
}Output:
public class Test {
private final String example = "example";
private final boolean isTestCode = true;
}It is possible to add custom tags. There a two types of tags: SingleTag that works on one line only and RangeTag that works on a range of lines. Tags are defined as follows:
classDiagram
Tag <|-- SingleTag
SingleTag <|-- RangeOpenTag
SingleTag <|-- RangeCloseTag
Tag <|-- RangeTag
class Tag{
+offset: int
+start: int
+end: int
+is_valid()*: bool
+execute()*: Optional[str]
}
class SingleTag{
+regex: str
+param_start: int
+param_end: int
+regex_start: int
+regex_end: int
+leading_characters: str
+parameter: str
+whitespace: str
+SingleTag(data: TagData)
}
class RangeOpenTag{
+RangeOpenTag(parent: Type, data: TagData)
}
class RangeCloseTag{
+RangeCloseTag(parent: Type, data: TagData)
}
class RangeTag{
+inset: int
+start: int
+end: int
+open_tag: RangeOpenTag
+close_tag: RangeCloseTag
+tags: List[Tag]
+RangeTag(open_tag: RangeOpenTag, close_tag: RangeCloseTag)
+add_tags(tags: Iterable[Tag])
}
The idea is that every tag has the following methods:
is_valid: whether the tag is validexecute: handle the text for this tag
RangeTag work in the following way:
RangeOpenTag: Specifies the open regex and handles the opening line. Defines the type of parent it belongs to (so that the tokenizer can match open and close tag)RangeCloseTag: Specifies the close regex and handles the closing line. Defines the type of parent it belongs to (so that the tokenizer can match open and close tag)RangeTag: Handles lines in between the open and close tag. Has access to both the open and close tag that were matched by the tokenizer.
Create a custom tag:
- Create a new file for your tag(s)
- Depending on if you create a
SingleTagorRangeTag
- SingleTag:
class TestTag(SingleTag):
regex = r'<regex>' # Regex that should match the tag
def __init__(self, data: TagData) -> None:
super().__init__(data)
def execute(self, content: str) -> Optional[str]:
# Manipulate the line
# None means the line is removed
def is_valid(self) -> bool:
# Return whether the tag is valid- RangeTag: Range needs a
RangeOpenTag,RangeCloseTagand aRangeTag
class TestOpenTag(RangeOpenTag):
regex = r'<regex>' # Regex that should match the open tag
def __init__(self, data: TagData) -> None:
super().__init__(TestRangeTag, data)# Type of RangeTag is belong to
def execute(self, content: str) -> Optional[str]:
# Manipulate the line
# None means the line is removed
def is_valid(self) -> bool:
# Return whether the tag is valid
class TestCloseTag(RangeCloseTag):
# Same as RangeOpenTag
class TestRangeTag(RangeTag):
regex = None # The matching is done based on open/close tag
def __init__(self, open_tag: RangeOpenTag, close_tag: RangeCloseTag):
super().__init__(open_tag, close_tag)
def execute(self, content: str) -> Union[str, None]:
# Manipulate lines between the tags- Export the new tag(s) in
codestripper/tags/__init__.py, import them in thetokenizerand add them to itsdefault_tags,
default_tags: Set[Type[SingleTag]] = {
IgnoreFileTag,
RemoveOpenTag,
...,
TestTag,
TestOpenTag,
TestCloseTag
}
⚠️ Only theSingleTag(s) (includingRangeOpenTagandRangeCloseTag) need to be added, not theRangeTag
Releases are automated with semantic-release. Pull requests are squash merged, so the PR title becomes the commit on main and must follow Conventional Commits (checked on every PR):
| PR title | Release |
|---|---|
fix: ..., perf: ... |
patch (1.2.3 → 1.2.4) |
feat: ... |
minor (1.2.3 → 1.3.0) |
! after the type (e.g. feat!: ..., refactor!: ...) or a BREAKING CHANGE: footer |
major (1.2.3 → 2.0.0) |
docs:, chore:, ci:, build:, refactor:, test:, style:, revert: |
no release |
On every merge to main the next version is determined, tagged (vX.Y.Z), a GitHub release is created and the package is published to PyPI. The version is set during the build and is not committed, so the version in pyproject.toml is not the released version.