Add BCn encoders/decoders with RDO support

.gitignore

@@ -58,3 +58,6 @@ the_khronos_group_inc.p12
 
 # Place for random developer test files.
 /testground/
+
+# clangd compile commands (used by LSPs)
+compile_commands.json

.gitmodules

@@ -2,7 +2,8 @@
 # Copyright 2022-2023 RasterGrid Kft.
 # SPDX-License-Identifier: Apache-2.0
 
+# TODO: revert this before merging!!
 [submodule "tests/cts"]
 	path = tests/cts
-	url = https://github.com/KhronosGroup/KTX-Software-CTS.git
+	url = https://github.com/walcht/KTX-Software-CTS.git
 	branch = main

REUSE.toml

@@ -119,6 +119,12 @@ precedence = "aggregate"
 SPDX-FileCopyrightText = "1997-2022 Sam Lantinga <slouken@libsdl.org>"
 SPDX-License-Identifier = "Zlib"
 
+[[annotations]]
+path = "external/bc7enc_rdo/**"
+precedence = "aggregate"
+SPDX-FileCopyrightText = "2020 Richard Geldreich <richgel99@gmail.com>"
+SPDX-License-Identifier = "MIT"
+
 [[annotations]]
 path = "cmake/modules/FindVulkan.cmake"
 precedence = "aggregate"

external/bc7enc_rdo/.gitrepo

@@ -0,0 +1,12 @@
+; DO NOT EDIT (unless you know what you are doing)
+;
+; This subdirectory is a git "subrepo", and this file is maintained by the
+; git-subrepo command. See https://github.com/ingydotnet/git-subrepo#readme
+;
+[subrepo]
+	remote = https://github.com/KhronosGroup/bc7enc_rdo.git
+	branch = changes_for_ktx
+	commit = dbe416d28a5530b4e8cc45b14bf034dc6b96bbde
+	parent = 68bb0de88c0c61dc4b34326df20ea9a6ed5f9646
+	method = merge
+	cmdver = 0.4.9

external/bc7enc_rdo/LICENSE

@@ -0,0 +1,68 @@
+If you use this software in a product, attribution / credits is requested but not required.
+
+bc7e.ispc uses the Apache 2.0 license and is Copyright (C) 2018-2021 Binomial LLC.
+LodePNG is Copyright (c) 2005-2016 Lode Vandevenne. See LodePNG.cpp for its license.
+
+All other source code files in this repo are available under 2 licenses -- choose whichever you prefer.
+
+ALTERNATIVE A - MIT License
+Copyright(c) 2020-2021 Richard Geldreich, Jr.
+Permission is hereby granted, free of charge, to any person obtaining a copy of
+this software and associated documentation files(the "Software"), to deal in
+the Software without restriction, including without limitation the rights to
+use, copy, modify, merge, publish, distribute, sublicense, and / or sell copies
+of the Software, and to permit persons to whom the Software is furnished to do
+so, subject to the following conditions :
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+------------------------------------------------------------------------------
+ALTERNATIVE B - Public Domain(www.unlicense.org)
+This is free and unencumbered software released into the public domain.
+Anyone is free to copy, modify, publish, use, compile, sell, or distribute this
+software, either in source code form or as a compiled binary, for any purpose,
+commercial or non - commercial, and by any means.
+In jurisdictions that recognize copyright laws, the author or authors of this
+software dedicate any and all copyright interest in the software to the public
+domain.We make this dedication for the benefit of the public at large and to
+the detriment of our heirs and successors.We intend this dedication to be an
+overt act of relinquishment in perpetuity of all present and future rights to
+this software under copyright law.
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE
+AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+------------------------------------------------------------------------------
+
+LodePNG version 20161127
+
+Copyright (c) 2005-2016 Lode Vandevenne
+
+This software is provided 'as-is', without any express or implied
+warranty. In no event will the authors be held liable for any damages
+arising from the use of this software.
+
+Permission is granted to anyone to use this software for any purpose,
+including commercial applications, and to alter it and redistribute it
+freely, subject to the following restrictions:
+
+	 1. The origin of this software must not be misrepresented; you must not
+	 claim that you wrote the original software. If you use this software
+	 in a product, an acknowledgment in the product documentation would be
+	 appreciated but is not required.
+
+	 2. Altered source versions must be plainly marked as such, and must not be
+	 misrepresented as being the original software.
+
+	 3. This notice may not be removed or altered from any source
+	 distribution.
+

external/bc7enc_rdo/README.md

@@ -0,0 +1,165 @@
+bc7enc - Fast BC1-7 GPU texture encoders with Rate Distortion Optimization (RDO)
+
+*Note: An even faster alternative to bc7e.ispc is now available - see [bc7f](https://richg42.blogspot.com/2026/01/bc7f-new-real-time-analytical-bc7.html) (or [here](https://github.com/BinomialLLC/basis_universal/wiki/Transcoder-Internals-Analytical-Real-Time-Encoders)). We've integrated bc7f into this repo for testing, and when we get the time we'll release the integration here.*
+
+This repo contains fast texture encoders for BC1-7. All formats support a simple post-processing transform on the encoded texture data designed to trade off quality for smaller compressed file sizes using LZ compression. Significant (10-50%) size reductions are possible. The BC7 encoder also supports a "reduced entropy" mode using the -e option which causes the output to be biased/weighted in various ways which minimally impact quality, which results in 5-10% smaller file sizes with no slowdowns in encoding time.
+
+Currently, the entropy reduction transform is tuned for Deflate, LZHAM, or LZMA. The method used to control the rate-distortion tradeoff is the classic Lagrangian multiplier RDO method, modified to favor MSE on very smooth blocks. Rate is approximated using a fixed Deflate model. The post-processing transform applied to the encoded texture data tries to introduce the longest match it can into every encoded output block. It also tries to continue matches between blocks and (specifically for codecs like LZHAM/LZMA/Zstd) it tries to utilize REP0 (repeat) matches.
+
+You can see examples of the RDO BC7 encoder's current output [here](https://richg42.blogspot.com/2021/02/more-rdo-bc7-encoding.html). Some examples on how to use the command line tool are on my blog, [here](https://richg42.blogspot.com/2021/02/how-to-use-bc7encrdo.html).
+
+This repo contains both [bc7e.ispc](https://github.com/BinomialLLC/bc7e) and its distantly related but weaker 4 mode only non-ispc variant, bc7enc.cpp. By default, if you set SUPPORT_BC7E=TRUE when running cmake, you get bc7e.ispc, otherwise you get bc7enc.cpp. (The -C option forces bc7enc.cpp.) bc7e supports all BC7 modes and features, but doesn't yet support reduced entropy BC7 encoding. bc7enc.cpp supports optional reduced entropy encoding (using -e with the command line tool). RDO BC7 is supported when using either encoder, however.
+
+The next major focus will be improving the default smooth block handling and improving rate distorton performance.
+
+This repo was originally derived from [bc7enc](https://github.com/richgel999/bc7enc) and [bc7e](https://github.com/BinomialLLC/bc7e). Note this repo contains the latest version of bc7e.ispc, which has a determinism bug fix.
+
+**Note: If you use this software in a product, attribution / credits is requested but not required. Thanks!**
+
+### Compiling
+
+This build has been tested with MSVC 2019 x64 and clang 6.0.0 under Ubuntu v18.04.
+
+To compile with bc7e.ispc (on Linux this requires [Intel's ISPC compiler](https://ispc.github.io/downloads.html) to be in your path - recommended):
+
+```
+cmake -D SUPPORT_BC7E=TRUE .
+make
+```
+
+To compile without BC7E:
+
+```
+cmake .
+make
+```
+
+Note the MSVC and Linux builds enable OpenMP for faster compression.
+
+### Examples
+
+The [.DDS](https://docs.microsoft.com/en-us/windows/win32/direct3ddds/dx-graphics-dds-pguide) output files can be loaded/viewed using tools like [AMD Compressonator](https://gpuopen.com/compressonator/).
+
+To encode to non-RDO BC7 using BC7E, highest quality, linear RGB(A) metrics:
+
+```
+./bc7enc blah.png
+```
+
+To encode to non-RDO BC7 using BC7E, highest quality, using perceptual (scaled YCbCr) colorspace error metrics:
+
+```
+./bc7enc blah.png -s
+```
+
+To encode to RDO BC7 using BC7E, highest quality, lambda=.5, linear metrics (perceptual colorspace metrics are always automatically disabled when -z is specified), with a balance of encoding performance vs. RDO efficiency:
+
+```
+./bc7enc blah.png -z.5
+```
+
+To encode to RDO BC7 using BC7E, lower baseline quality (-u4) for faster encoding, lambda=.5, and with faster encoding (only inject one match vs two, with a tiny RDO lookback window size of 16 bytes):
+
+```
+./bc7enc blah.png -u4 -z.5 -ze -zc16
+```
+
+To encode to non-RDO BC7 using entropy reduced or quantized/weighted BC7 (no slowdown vs. non-RDO bc7enc.cpp for BC7, slightly reduced quality, but 5-10% better LZ compression, only uses 2 or 4 BC7 modes):
+
+```
+./bc7enc blah.png -C -e
+```
+
+To encode to RDO BC7 using the entropy reduction transform combined with reduced entropy BC7 encoding, with a slightly larger window size than the default which is 128 bytes:
+
+```
+./bc7enc -zc256 blah.png -C -e -z1.0
+```
+
+Same as before, but higher compression (allow 2 matches per block instead of 1):
+
+```
+./bc7enc -zc256 blah.png -C -e -z1.0 -zn
+```
+
+Same, except disable ultra-smooth block handling:
+
+```
+./bc7enc -zc256 blah.png -C -e -z1.0 -zu
+```
+
+To encode to RDO BC7 using the entropy reduction transform at lower quality, combined with reduced entropy BC7 encoding, with a slightly larger window size than the default which is 128 bytes:
+
+```
+./bc7enc -zc256 blah.png -C -e -z2.0
+```
+
+To encode to RDO BC7 using the entropy reduction transform at higher effectivenes using a larger window size, without using reduced entropy BC7 encoding:
+
+```
+./bc7enc -zc1024 blah.png -z1.0
+```
+
+To encode to RDO BC7 using the entropy reduction transform at higher effectivenes using a larger window size, with a manually specified max smooth block max error scale:
+
+```
+./bc7enc -zc1024 blah.png -z2.0 -zb30.0
+```
+
+To encode to RDO BC7 using the entropy reduction transform at higher effectivenes using a larger window size, using only mode 6 (more block artifacts, but better rate-distortion performance as measured by PSNR):
+
+```
+./bc7enc -zc1024 blah.png -6 -z1.0 -e
+```
+
+To encode to BC1:
+```
+./bc7enc -1 blah.png
+```
+
+To encode to BC1 with Rate Distortion Optimization (RDO) at lambda=1.0:
+```
+./bc7enc -1 -z1.0 blah.png
+```
+
+The -z option controls lambda, or the rate vs. distortion tradeoff. 0 = maximum quality, higher values=lower bitrates but lower quality. Try values [.25-8].
+
+To encode to BC1 with RDO, with RDO debug output, to monitor the percentage of blocks impacted:
+```
+./bc7enc -1 -z1.0 -zd blah.png
+```
+
+To encode to BC1 with RDO with a higher then default smooth block scale factor:
+```
+./bc7enc -1 -z1.0 -zb40.0 blah.png
+```
+
+Use -zb1.0 to disable smooth block error scaling completely, which increases RDO performance but can result in noticeable artifacts on smooth/flat blocks at higher lambdas.
+
+Use -zc# to control the RDO window size in bytes. Good values to try are 16-8192. 
+Use -zt to disable RDO multithreading. 
+
+To encode to BC1 with RDO at the highest achievable quality/effectiveness (this is extremely slow):
+
+```
+./bc7enc -1 -z1.0 -zc32768 blah.png
+```
+
+This sets the window size to 32KB (the highest setting that makes sense for Deflate). Window sizes of 2KB (the default) to 8KB are way faster and in practice are almost as effective. The maximum window size setting supported by the command line tool is 64KB, but this would be very slow.
+
+For even higher quality per bit (this is incredibly slow):
+```
+./bc7enc -1 -z1.0 -zc32768 -zm blah.png
+```
+
+### Dependencies
+There are no 3rd party code or library dependencies. utils.cpp/.h is only needed by the example command line tool. It uses C++11. The individual .cpp files are designed to be easily dropped into other codebases.
+
+For RDO post-processing of any block-based format: ert.cpp/.h. You provide this function an array of encoded blocks, an array of source/original 32bpp blocks, some parameters, and a pointer to a block decoder function for your format as a callback. It must return false if the passed in block data is invalid. (Make sure you *really* validate the block's data, because the ERT post-processor will inevitably call your callback with invalid blocks.) This transform works on most other texture formats, such as ETC1/2, EAC, and ASTC. The ERT works on block sizes ranging from 1x1 to 12x12. This file has no other dependencies apart from utils.cpp/h.
+
+For BC1-5 encoding/decoding: rgbcx.cpp/.h
+
+For BC7 encoding: bc7enc.cpp/.h
+
+For BC7 decoding: bc7decomp.cpp/.h
+