# texosquery Cross-platform Java application to query OS information designed for use in TeX's shell escape mechanism. ## Licence This material is subject to the LaTeX Project Public License. See http://www.ctan.org/license/lppl1.3 for the details of that license. Copyright 2016-2020 Nicola Talbot ## Description The application can query the following: - locale information - default file encoding - current working directory - user home directory - temporary directory - OS name, arch and version - Current date and time in PDF format (for TeX formats that don't provide `\pdfcreationdate`) - Date-time stamp of a file in PDF format (for TeX formats that don't provide `\pdffilemoddate`) - Size of a file in bytes (for TeX formats that don't provide `\pdffilesize`) - Contents of a directory (captured as a list) - Directory contents filtered by regular expression (captured as a list) - URI of a file - Canonical path of a file All paths use a forward slash as directory divider so results can be used, for example, in commands like `\includegraphics`. There are files provided for easy access in (La)TeX documents: - `texosquery.tex` : generic TeX code - `texosquery.sty` : LaTeX package This provides the command `\TeXOSQuery` to run `texosquery` using TeX's shell escape mechanism and capture the result in a control sequence. The category code of most of TeX's default special characters (and some other potentially problematic characters) is temporarily changed to 12 while reading the result. Some of the `texosquery` output contains short markup commands (such as `\wrp`) which are internally converted within `\TeXOSQuery`. ## Important Notes The TeX code uses a piped shell escape to capture the result from `texosquery`. This means that you must have the piped shell escape enabled to make use of this feature. MiKTeX users need the `--enable-pipes` option on the TeX command line to enable this. The alternative is to run `texosquery` outside of TeX and capture the output in a temporary file, which can then be read using `\TeXOSQueryFromFile`. You need to correctly set up the configuration file `texosquery.cfg` to match your system. The TeX package managers can't do this for you automatically. Copy the file to either your `TEXMFHOME` or `TEXMFLOCAL` tree to prevent it from being overwritten by subsequent updates. `texosquery-jre8` is on the restricted list for TeX Live 2017. To take advantage of this, you must have at least Java 8 installed and edit the `texosquery.cfg` file to: ```tex \def\TeXOSInvokerName{texosquery-jre8} \TeXOSQueryAllowRestricted ``` (Note that the second line above has been uncommented.) Note that due to TeX's security measures, quotes can't be used in the restricted mode's shell escape. This means that if you need to pass a file name containing a space as an argument to `texosquery`, you'll have to use the unrestricted mode. ## Installation Installation is best done using your TeX package manager. Manual installation instructions are described below. For more detail, see the documentation. ### Compiling the Documentation To compile the documentation: ```bash pdflatex texosquery.dtx makeglossaries texosquery makeindex -s gglo.ist -t texosquery.glg -o texosquery.gls texosquery.glo pdflatex texosquery.dtx makeindex -s gind.ist texosquery.idx pdflatex texosquery.dtx pdflatex texosquery.dtx ``` ### Extracting the Files Except for the `.jar` files, the `texosquery` files are all bundled inside `texosquery.dtx` with the extraction commands provided in `texosquery.ins`. To extract all the files do: ```bash tex texosquery.ins ``` The `.sh` files are bash scripts for Unix-like users. These will need to be set to executable using `chmod`. I recommend that you also remove the `.sh` extension. Windows users can deleted these files. The `.batch` files are batch scripts for Windows users. Unix-like users can delete these files. These files are given the extension `.batch` as TeX on Windows doesn't allow the creation of `.bat` files. The extension will need to be changed to `.bat` after the files have been extracted. ### Installing the Application In the following, replace _TEXMF_ with the path to your TEXMF directory. You can find your home TEXMF directory using ```bash kpsewhich -var-value=TEXMFHOME ``` There are now three Java applications provided by this package: - `texosquery.jar` (requires at least Java 7) - `texosquery-jre8.jar` (requires at least Java 8) - `texosquery-jre5.jar` (requires at least Java 5) (See the ["Security"](#security) section below.) There are corresponding bash scripts for Unix-like users (the `.sh` extension added by `texosquery.ins` should be removed and the files made executable): - `texosquery` - `texosquery-jre8` - `texosquery-jre5` There are also corresponding batch scripts for Windows users (the `.batch` extension created by the `.ins` file should be changed to `.bat`): - `texosquery.bat` - `texosquery-jre8.bat` - `texosquery-jre5.bat` Each script uses `kpsewhich` to find the corresponding `.jar` file and runs it. Put the `.jar` files in _TEXMF_`/scripts/texosquery/` and the bash or `.bat` files somewhere on your path. You may or may not need to refresh the TeX database. To test the installation, run ```bash texosquery -v ``` in the command prompt or terminal. (Also try with `texosquery-jre8` and `texosquery-jre5`.) If successful, it should show the version number. ### Installing the TeX Code The `.tex`, `.cfg` and `.sty` files should all be extracted from `texosquery.dtx` with `tex texosquery.ins` as described above. - Move `texosquery.tex` to _TEXMF_`/tex/generic/texosquery/` - Move `texosquery.cfg` to _TEXMF_`/tex/generic/texosquery/` - Move `texosquery.sty` to _TEXMF_`/tex/latex/texosquery/` - Move `texosquery.pdf` to _TEXMF_`/doc/generic/texosquery/` The `texosquery.cfg` file allows you to specify which application you want to use. First check which version of the Java Runtime Environment (JRE) you have installed: ```bash java -version ``` This should display the version information. (For example, `"1.8.0_92"`) If the version number starts with `1.8` ("Java 8"), then you can use `texosquery-jre8.jar`. This is the full application. You can use `texosquery` or `texosquery-jre5` as well, but there's less locale support with them. The `texosquery-jre8` bash script and `texosquery-jre8.bat` batch script invokes Java with `-Djava.locale.providers=CLDR,JRE`. If Windows users have simply been supplied with an executable versions of the `texosquery-jre8.jar` file (`texosquery-jre8.exe`) then you'll have to set the `JAVA_TOOL_OPTIONS` environment variable instead. This `java.locale.providers` setting allows Java to access locale information from the [Unicode Common Locale Data Repository (CLDR)](http://cldr.unicode.org/) installed on their system. The pending Java 9 should include this by default, so this will only be relevant to Java 8. Earlier versions of Java (7 or below) don't have this option, which limits the locale support to that which is natively provided by the JRE. Note that Java 7 and earlier versions [have reached their end of life](http://www.oracle.com/technetwork/java/eol-135779.html) and are now **deprecated**. If the version information starts with `1.7` ("Java 7"), then you can use `texosquery.jar`. This is the default application and provides most of the functions of the full application, but there's less locale support. You can also use the even more limited `texosquery-jre5`, but you can't use `texosquery-jre8`, so you can't take advantage of the CLDR. If the version information starts with `1.5` ("Java 5") or `1.6` ("Java 6"), then you can _only_ use `texosquery-jre5`. The locale support is significantly reduced in this case and there's no support for language scripts. Once you have determined which application you want to use, edit the `texosquery.cfg` so that `\TeXOSInvokerName` is defined to the appropriate invocation. For example, with Java 8: ```tex \def\TeXOSInvokerName{texosquery-jre8} ``` or with Java 5 or 6: ```tex \def\TeXOSInvokerName{texosquery-jre5} ``` (bash users will have to add the `.sh` extension if it hasn't been removed, as per the above instructions.) ## Examples: Plain TeX: ```tex % arara: pdftex: {shell: on} \input texosquery \TeXOSQuery{\result}{-b} IEFT BCP 47 language tag: \result. \TeXOSQueryLocale{\result} POSIX locale: \result. \TeXOSQueryCwd{\result} cwd: {\tt \result}. \TeXOSQueryNow{\result} now: \result. \TeXOSQueryFileDate{\result}{\jobname.tex} file date: \result. \TeXOSQueryFileSize{\result}{\jobname.tex} file size: \result bytes. \bye ``` LaTeX: ```latex % arara: pdflatex: {shell: on} \documentclass{article} \usepackage{texosquery} \usepackage{etoolbox} \begin{document} \TeXOSQuery{\result}{-b} IEFT BCP 47 language tag: \result. \TeXOSQueryLocale{\result} POSIX locale: \result. \TeXOSQueryCwd{\result} cwd: \texttt{\result}. \TeXOSQueryNow{\result} now: \result. \TeXOSQueryFileDate{\result}{\jobname.tex} file date: \result. \TeXOSQueryFileSize{\result}{\jobname.tex} file size: \result bytes. \TeXOSQueryFilterFileList{\result}{,}{.+\string\.(png|jpg)}{.} All jpg and png files in current directory: \renewcommand{\do}[1]{\texttt{#1}.\par} \expandafter\docsvlist\expandafter{\result} \end{document} ``` For a full list of available commands, see the documentation (`texosquery.pdf`). ## Troubleshooting If something goes wrong with the call to `texosquery`, the control sequence will be set to empty. If this happens, here are the steps to diagnose the problem: 1. Check the log file for any lines starting with `TeXOSQuery:` For example: `TeXOSQuery: texosquery-jre8 --pdfnow` If found, this means that the dry run mode was on and the shell escape wasn't used. Ensure that the shell escape is enabled when you build the document. 2. If the log file contains `(|texosquery` _options_`)` then the shell escape was used but `texosquery` returned an empty value. In this case, copy the part between `(|` and `)` and paste it into a command prompt or terminal but insert `--debug` at the start of the options list. For example, if the log file contains `(|texosquery-jre8 --cwd)` then run ```sh texosquery-jre8 --debug --cwd ``` This should now display an error message explaining the problem. (For example, read access forbidden or file not found or a security exception.) ## Security As from version 1.2, all variants of `texosquery` (JRE5, 7 and 8) obey TeX Live's `openin_any` setting. Any of the actions that involve reading file information won't work if read access is forbidden by `openin_any` or by the operating system. MiKTeX doesn't use the `openin_any` setting, so if not set `texosquery` will assume `a` (any). In addition to obeying `openin_any`, the file listing actions (such as `--list`) for the JRE7 and 8 variants also prohibit listing the contents outside of the current working directory's path even if `openin_any=a`. This means that you can't, for example, list the contents of `..` (the current working directory's parent) nor can you try walking the entire file system. The `--walk` action additionally won't descend hidden directories. This extra restriction is designed to prevent malicious code from trying to use `texosquery` to look around your filing system. The Java 5 version `texosquery-jre5` is the least secure and doesn't have this additional restriction on the file listing actions. However, it still obeys `openin_any`. The `--walk` action is not available with `texosquery-jre5`. Examples: ```bash texosquery-jre8 --debug --walk ',' '.*' / ``` This action is forbidden: ``` texosquery-jre8: Can't walk directory: / ``` Changing `/` to `.` is allowed, but not for `texosquery-jre5`: ```bash texosquery-jre5 --debug --walk ',' '.*' . ``` returns: ``` texosquery-jre5: walk requires at least JRE 7 version ``` With `openin_any=a`, the `--userhome` action is usually allowed (depends on the operating system and Java's security manager). ```bash texosquery-jre8 --debug --userhome ``` returns ``` \fslh home\fslh nlct ``` (which `\TeXOSQuery` converts to `/home/nlct`.) However, unless this also happens to be the current working directory, it's not possible to obtain a file listing: ```bash texosquery-jre8 --debug --list ',' /home/nlct ``` returns ``` texosquery-jre8: Unable to list contents of: /home/nlct texosquery-jre8: Listing outside cwd path not permitted: /home/nlct ``` ## Source code The Java source is in the `.java` files and the manifest for each `.jar` file is `Manifest*.txt` (`Manifest-jre8.txt` for `texosquery-jre8` etc). Assuming the following directory structure: ``` java/TeXOSQuery.java java/TeXOSQueryJRE5.java java/TeXOSQueryJRE7.java java/TeXOSQueryJRE8.java java/QueryAction.java java/QueryActionType.java java/FileSortType.java java/FileSortComparator.java java/FileListType.java java/FileWalkVisitor.java java/Manifest-jre5.txt java/Manifest-jre7.txt java/Manifest-jre8.txt classes/com/dickimawbooks/texosquery/ ``` Then to create `texosquery-jre8.jar`, do (for JDK version 1.8): ```bash cd java javac -d ../classes TeXOSQuery.java QueryAction.java QueryActionType.java TeXOSQueryJRE8.java FileSortType.java FileSortComparator.java FileListType.java FileWalkVisitor.java cd ../classes jar cmf ../java/Manifest-jre8.txt ../texosquery-jre8.jar com/dickimawbooks/texosquery/*.class ``` Similarly for the other `.jar` files, replacing `8` with `7` or `5`. (The Java 7 version should really be `texosquery-jre7.jar`, but is simply named `texosquery.jar` in the distribution for backward compatibility reasons. If you compile the code, the `.jar` name is your choice.) --- Source on GitHub: https://github.com/nlct/texosquery Author Home Page: http://www.dickimaw-books.com/