Qore xml module
===============

INTRODUCTION
------------
The Qore xml module provides comprehensive XML functionality to the Qore
Programming Language. This module was previously part of the main qore library
but was separated into a standalone module in Qore v0.8.1.

The module is built on libxml2, providing a powerful, stable, and thread-safe
basis for XML integration in Qore.

See the HTML documentation in docs/ for detailed API reference and examples.


FEATURES
--------
Core XML Processing:
- XML serialization (Qore data structures to XML strings)
- XML deserialization (XML strings to Qore data structures)
- DOM document parsing and manipulation (XmlDoc, XmlNode classes)
- XPath query support
- Streaming XML parsing (XmlReader, SaxIterator classes)
- XML validation (XSD Schema, RelaxNG, DTD)
- Native XSD document ID/IDREF validation with forward references, selected union
  values and validation-root scope; see design/native-id-bindings.md
- Native integer, decimal, boolean and binary default/fixed declarations require valid
  canonical spellings, including list/union members; see design/native-numeric-defaults.md
- Native float/double conversion preserves IEEE values independently of locale and
  rounding mode, with canonical constraint checks; see design/native-ieee-constraints.md

Web Services:
- SOAP client and server implementations
- WSDL parsing and processing
- XSD element wildcard assessment, ordered output, providers and retained XML
  values; see design/wsdl-element-wildcards.md
- Generic XSD values with mixed XML content, namespace preservation and validating
  providers; see design/wsdl-generic-values.md
- Validated element default/fixed declarations with typed, namespace-aware metadata;
  see design/wsdl-element-constraints.md
- Empty WSDL elements apply canonical defaults under the actual type; opt-in native
  values retain empty occurrence state through saved providers and SOAP consumers;
  see design/wsdl-element-defaults.md
- Native calendar validation and canonical declarations preserve exact fractional seconds and
  extended years; see design/native-calendar-constraints.md
- Native empty-element defaults use canonical actual-type values and declaration QName bindings;
  validation preserves the input XML; see design/native-element-defaults.md
- Canonical numeric, boolean, binary, IEEE and date/time default/fixed declaration checks preserve selected
  values through saved schemas and providers; see design/wsdl-canonical-constraints.md
- Receiving-element nil validation with attribute-preserving native values; see
  design/wsdl-element-nil.md
- Ordered native mixed text and typed children through schema, SOAP and provider
  consumers; see design/wsdl-mixed-content.md
- XML-RPC client and server support, with exact string/struct-name character data
  and UTF-16 serialization; see design/xmlrpc-character-data.md
- Salesforce API client support

Additional Features:
- WebDAV protocol handler
- Data provider integration (SAX, SOAP data providers)
- Custom I/O callbacks for external resource resolution

Bundled User Modules:
- WSDL - Web Services Description Language parser
- SoapClient - SOAP client implementation
- SoapHandler - SOAP server request handler
- SalesforceSoapClient - Specialized Salesforce SOAP client
- XmlRpcHandler - XML-RPC server implementation
- XmlRpcConnection - XML-RPC connection management
- WebDavHandler - WebDAV protocol handler
- SaxDataProvider - SAX-based data provider
- SoapDataProvider - SOAP-based data provider
- CargoXmlDataProvider - Cargo XML data provider for B2B logistics
  (UN/CEFACT Cross-Industry messages bundled; pluggable for IATA Cargo-XML)


LICENSE
-------
The source code is released under dual licenses: LGPL 2.1 and MIT. Either
license may be used at the user's discretion. Both licenses allow the module
to be loaded without restrictions by the Qore library (even when the Qore
library is initialized in GPL mode).

See COPYING.MIT and COPYING.LGPL for details on the open-source licenses.


REQUIREMENTS
------------
- Qore 3.0+ with resolve_url() and HTTP effective-URL metadata (https://qore.org)
- libxml2 2.6.0+ (http://www.xmlsoft.org)
- OpenSSL (for HTTPS support)
- C++17 compatible compiler
- CMake 3.18+ (for CMake builds)


BUILDING WITH CMAKE (Recommended)
---------------------------------
mkdir build && cd build
cmake ..
make
make install

CMake options:
  -DCMAKE_INSTALL_PREFIX=<path>  Installation prefix
  -DCMAKE_BUILD_TYPE=Release     Build type (Release, Debug, RelWithDebInfo)
  -DQORE_XML_LIBXML2_PROVIDER=AUTO  Probe system libxml2; fetch a fixed version if needed
  -DQORE_XML_LIBXML2_PROVIDER=SYSTEM  Require the system library to pass the probe
  -DQORE_XML_LIBXML2_PROVIDER=BUNDLED  Always build the pinned private dependency

CMake runs namespace-URI, XSD QName, ENTITY and schema-resource URI regression probes against the installed library.
Affected versions (including unpatched 2.12.10) incorrectly decode ampersands in
namespace declarations. A passing distribution backport is accepted regardless
of its version string. AUTO uses FetchContent when the library is missing or the
probe fails. The fallback is libxml2 2.15.4, the latest stable release verified on
2026-09-08, pinned by URL and SHA-256; it is statically linked with hidden symbols
into the XML module, with upstream notices installed under share/licenses/qore-xml.
It does not replace or install a system libxml2. Maintainers
update the pin explicitly, so an unchanged checkout has reproducible downloads.
The bundled build also frees temporary catalog error strings before restoring
libxml2's prior error record. This one-line ownership correction is compiled
from a verified build-tree copy; downloaded and offline source trees remain
unchanged. A source override with a different parser implementation must already
contain the correction, otherwise configuration fails explicitly.
The probe also verifies implicit xml-prefix resolution and QName equality across
absent and explicitly empty default namespaces, using both DOM and streaming XSD
validation, and verifies ordered union matching after a QName candidate rejects
an unbound prefix. Unpatched 2.15.4 fails these checks. The bundled build corrects the
namespace lookup, value comparison and union trial error reporting in verified
build-tree copies of
xmlschemas.c and xmlschemastypes.c. Both original and corrected source hashes are
checked; unrelated source overrides fail configuration. System backports that
pass the complete probe remain eligible for SYSTEM or AUTO selection.
The particle checks reject competing local/global references, shared group uses,
all members and wildcard alternatives while preserving repeated emissions of one
source position. The private dependency retains position identity independently
of callback declarations; see design/xml-particle-identity.md.
Counted attribution checks every present group before automaton simplification,
permits exact fixed boundaries, and retains nested counter increments during
validation. Nullable repetitions do not enumerate empty iterations. The complete
behavior probe includes DOM and reader validation; see
design/xml-particle-attribution.md. Finite occurrence bounds retain their exact
values through parsing, compilation and execution, including values above native
integer ranges; see design/xml-particle-ranges.md. The probe also checks large
finite, nullable and inherited ranges before selecting a system backport.
Anonymous simple types inherit their source schema's finalDefault. The behavior
probe checks restriction, list and union exclusions, including unrelated and
absent defaults; the private dependency corrects the missing anonymous-type
initialization. See design/wsdl-type-final.md and test/xml-type-final.qtest.
Element substitution checks include builtin restriction, union membership and
both orders of mixed complex derivation. The private dependency counts every
derivation method before applying the head and intermediate type blocks.
The 45-schema probe exercises both DOM and reader validation; a backport missing
these corrections fails SYSTEM selection and causes AUTO to use the private
dependency. See design/wsdl-element-substitution.md and
test/xml-element-substitution.qtest.
Native schema construction also checks explicit and implicit element declaration
consistency, including unused groups. An additional 34-schema probe detects
missing system checks; AUTO selects the corrected private dependency. See
design/native-element-consistency.md for the contract and an order example.
Native DOM and streaming validation reject multiple assessed wildcard ID attributes
and wildcard IDs when the complex type declares an ID attribute use, even if it
is absent. ID restrictions follow the base-type hierarchy. Schema default/fixed
constraints on referenced attributes are checked before their values are used.
The configure probe covers 180 documents and 100 ID ancestry schemas; see
design/native-wildcard-ids.md for behavior and independent-validator differences.
Strict element wildcards can assess an available xsi:type without a global element
declaration. Namespace constraints, selected-type facets and content still apply.
Schema declarations accept whitespace-only CDATA while preserving complete
annotation subtrees. CMake detects both behaviors before selecting system libxml2;
see design/native-wildcard-types.md for examples, provider selection and tests.
Native character assessment treats empty CDATA as zero characters and accepts
XML whitespace inside CDATA in element-only content. Nil, required attributes
and default/fixed rules remain enforced; CMake detects missing system behavior.
See design/native-character-content.md for the event and ownership contract.
Native fixed constraints compare typed values, including equivalent scalar and
list spellings. Clock-only time comparisons normalize timezone offsets without inventing
calendar dates. XSD 1.0 unsigned builtins reject signed lexical forms, while
nonNegativeInteger still permits signed zero. Integer, list-token, string-value
and URI allocation failures remain internal errors with complete cleanup.
Binary allocation failures follow the same rule. Native Base64 accepts only the
XSD alphabet and XML whitespace, rejecting ignored punctuation and other Unicode
spacing characters; see design/native-binary-constraints.md.
CMake probes these behaviors before choosing a provider; see
design/native-value-spaces.md for ownership, examples and test coverage.
Native anyType extensions share the same private particle layout as builtin
types, with immutable builtin occurrence metadata. Empty group references
retain their effective content and therefore require the proper mixed-content
derivation. CMake checks both paths; see design/native-anytype-particles.md.
WSDL complex-content extensions now inherit the same anyType wildcard particle.
Active wildcards in base types and named groups retain their XML content through
native providers, Serializable reconstruction and SOAP consumers. Empty local
compositors inherit the base content; mixed and ambiguous derivations are checked.
See design/wsdl-anytype-inheritance.md for the effective-particle contract.
Explicit nonempty element values enforce fixed constraints by computed XSD value
identity through saved schemas, providers and SOAP consumers, with constrained
provider examples. See design/element-fixed-values.md.
WSDL child particles admit concrete substitution members with shared occurrence
limits, member-specific conversion, provider fields and samples. Native fields
retain the selected member name; XsdXmlValue also retains interleaving and exact
XML. See design/wsdl-substitution-particles.md for an order/quantity example.

Complete schema elements and literal SOAP body/header parts retain selected
substitution roots using portable ^element^ / ^val^ wrappers. Native providers,
retained XML, samples and Serializable reconstruction preserve the selected
member. Part matching checks complete, unique ownership by expanded name.
See design/wsdl-substitution-roots.md and test/wsdl-substitution-roots.qtest.
Attribute wildcards enforce namespace constraints and strict/lax/skip processing
through schema, SOAP and provider values. Known attributes retain typed values;
unassessed attributes retain their expanded names, lexical text and namespace
context. Per-message namespace allocation preserves new partner attributes
without changing shared schema state. See design/wsdl-wildcard-attributes.md.
The URI checks cover raw Unicode/space schema locations, XML Base, reserved
percent escapes, empty path segments, dot-segment resolution and runtime schema
hints in DOM and streaming validation. The fallback applies checked URI fixes to
build-tree copies of uri.c, tree.c and the already QName-corrected xmlschemas.c.
The qore-xml-uri-allocation target separately checks allocation failures and recovery.
See design/xml-schema-uris.md and test/wsdl-interop/test_schema_uris.py for examples
and the exact original URI regression.
The ENTITY checks validate unparsed declarations in DOM, reader and SAX contexts,
including enumerations, lists, ordered unions and first entity bindings. The
private dependency records SAX declarations per document and constructs ENTITY
values before applying document constraints. Built-in ENTITIES, IDREFS and
NMTOKENS retain their minLength=1 facet during validation and restriction.
Schema construction rejects contradictory effective minimum and maximum lengths.
The qore-xml-entity-allocation target checks allocation failure and early-stop
cleanup. See design/xml-entity-validation.md for the native validation contract.

The private libxml2 build requires a C++17 compiler and standard library with
floating-point to_chars support. Decimal parsing uses the pinned, header-only
fast_float 8.3.0 implementation under the MIT license, including on standard
libraries that lack floating-point from_chars. Its IEEE conversion helper preserves caller
rounding modes, flags and traps; no additional Qore runtime dependency is added.
The upstream source root is included only by C sources that need private headers.
C++ sources must not search it: upstream VERSION aliases the standard <version>
header on case-insensitive filesystems, including default macOS APFS volumes.

The annotation probe checks foreign attributes by namespace, including foreign
lang attributes, and validates documentation source values as xs:anyURI without
fetching them. The private dependency fixes both checks; see
design/native-schema-annotations.md for the contract and examples.

The identity XPath probe checks full selector and field union paths. The private
dependency rejects empty paths and trailing union separators with checked cleanup;
legal token whitespace and explicit axes remain valid. See
design/native-identity-paths.md and test/wsdl-interop/identity-paths-evidence.md.

Component QName references collapse surrounding XML whitespace before namespace
and component lookup. CMake detects affected libraries; the private dependency
also preserves allocation failures from implicit xml namespace lookup. See
design/wsdl-identity-grammar.md for the native and WSDL declaration contracts.

Native key fields are checked against the assessed element declaration: a
nillable declaration is forbidden even when the field has a non-nil value.
Attribute fields on nillable elements remain valid. CMake detects the defect
and applies a guarded build-tree correction to the private dependency; see
design/native-key-nillable.md for the XSD 1.0 rule and examples.

Native unique/keyref constraints treat nil as a missing identity value under
the explicitly approved interpretation. Nil fields still count toward field
cardinality, and empty strings remain values. CMake probes these semantics;
see design/native-nil-identities.md and the interpretation record in
test/wsdl-interop/nil-identity-interpretation.md.

For offline builds, provide the unpacked pinned source with
  -DFETCHCONTENT_SOURCE_DIR_QORE_XML_LIBXML2=/path/to/libxml2-2.15.4
or provide a fixed system dependency and select SYSTEM. A failed download is a
configure error; CMake never silently uses a library that failed the probe.
Cross builds use CMAKE_CROSSCOMPILING_EMULATOR for the probe when available;
without an emulator AUTO selects the pinned fallback and SYSTEM fails clearly.
The selected dependency can be checked after building with
  cmake --build build --target qore-xml-namespace-probe
  build/qore-xml-namespace-probe

Autotools continues to use its explicitly selected system dependency; use a fixed
libxml2 with that build system. The automatic fallback is a CMake feature.

Example:
  cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_BUILD_TYPE=Release ..


BUILDING WITH AUTOTOOLS
-----------------------
To configure the build:
    ./configure --disable-debug

If the qore library cannot be found:
    ./configure --disable-debug --with-qore=<dir>

If libxml2 cannot be found:
    ./configure --disable-debug --with-libxml2-dir=<dir>

If openssl cannot be found:
    ./configure --disable-debug --with-openssl-dir=<dir>

The qore binary needs to be in the PATH so configure can determine the
module directory.

Then execute:
    make && make install

(or 'make && sudo make install' as needed)


TESTING
-------
Run the test suite:
    cd test
    qore xml.qtest -v
    qore soap.qtest -v

WSDL/SOAP interoperability regression tests (offline; from the repository root):
    qore --enable-debug test/wsdl-interop.qtest

See test/wsdl-interop/README.md for the W3C fixture sources, the optional full
corpus survey, and the remaining compatibility findings.


DOCUMENTATION
-------------
Full HTML documentation is generated with Doxygen:
    make docs

Documentation is generated in the docs/ directory.


SUPPORT
-------
Please direct questions to: david@qore.org

Bug reports and feature requests:
https://github.com/qorelanguage/qore/issues

Provider regression checks:
  python3 test/cmake/test_libxml2_provider.py -v
  qore --enable-debug test/xml-schema-callbacks.qtest
The CMake tests cover selection, offline and cross builds, install contents,
source immutability, and catalog-error allocation cleanup. Set XML_LIBXML2_SOURCE
to an unpacked pinned archive to run them without downloads. For local Qore tests,
set QORE_MODULE_DIR to the module build directory and repository qlib directory.

WSDL ENTITY/ENTITIES conversion checks selected unparsed-entity requirements at
XML instance boundaries, including union/list members, attributes and defaults.
SOAP messages cannot carry the required DTD declarations. See
`design/wsdl-entity-values.md` for datatype-only operations, document conversion,
examples and regression tests. Complete message providers also validate supplied
parts before acceptance; see `design/wsdl-message-providers.md`.

Default WSDL native sample generation validates the candidate before returning it
and reports XSD-SAMPLE-ERROR when generation cannot supply a valid instance.
See `design/wsdl-sample-instances.md` for errors, limits and explanatory options.

WSDL `xs:time` and `xs:dateTime` preserve missing timezones, arbitrary fractions,
extended years and leap seconds as validated strings when native dates would lose
information. Native values retain microseconds and the represented UTC offset.
See `design/wsdl-time-output.md` for conversion, exact facets and examples.

Native key/unique tables retain unambiguous child entries and prefer local entries
when inherited values conflict. Rejected child conflicts do not poison unrelated
ancestor entries. CMake detects affected system libraries with DOM/reader tests;
see design/native-identity-tables.md and test/xml-identity-tables.qtest.

Native identity fields also assess the four xsi attributes and preserve selected
list versus atomic values, including union members and defaults. Empty lists
remain values distinct from nil. Checked buffers preserve allocation failures in
identity formatting. See design/native-instance-identities.md and
test/xml-instance-identities.qtest.