The leader and body are delimited by the tokens.
| input | basic [text] to [be] scanned |
| find token pair | [ | | ] |
| check | leader | basic |
| check | body | text |
The next token pair is found.
| find token pair | [ | | ] |
| check | leader | to |
| check | body | be |
The remaining input is returned when no more tokens are found.
| find token pair | [ | | ] |
| check | leader | scanned |
| check | body | |
Empty input returns empty results.
| input | |
| find token pair | [ | | ] |
| check | leader | |
| check | body | |
Missing tokens returns empty body.
| input | no first token] |
| find token pair | [ | | ] |
| check | leader | no first token] |
| check | body | |
| input | no [second token |
| find token pair | [ | | ] |
| check | leader | no [second token |
| check | body | |
Out of order tokens returns empty body.
| input | out ]of[ order |
| find token pair | [ | | ] |
| check | leader | out ]of[ order |
| check | body | |
Adjacent tokens are ignored.
| input | scan [] this |
| find token pair | [ | | ] |
| check | leader | scan [] this |
| check | body | |
Empty leader is OK.
| input | [scan] |
| find token pair | [ | | ] |
| check | leader | |
| check | body | scan |
| find token pair | [ | | ] |
| check | leader | |
| check | body | |
Token body can be filtered.
| input | [not me] but [me] |
| find token pair | [ | | ] | filter | not me |
| check | leader | [not me] but |
| check | body | me |