All notable changes to this project will be documented in this file. See standard-version for commit guidelines.
0.13.1 (2018-12-10)
- optimize contextNormalPath mzgoddard/hard-source-webpack-plugin#469 (cb03413)
- replace native fs with graceful-fs (93143ca)
0.13.0 (2018-12-09)
- add prefix member to HarmonyExportExpression schema (#443) (fef1488)
- remove ExcludeMiniCssModulePlugin (f15bd9f)
- support new dll plugin schema changes (a250247)
0.12.0 (2018-07-30)
0.11.2 (2018-07-22)
- performance regression due to false position md5 (#413) (c670f73)
- serialize path query params without change (#408) (734e5b0)
- webpack 3 RuleSet creation (#407) (6551c8e)
0.11.1 (2018-07-12)
- plus sign missing (#398) (8885f10)
- serialize path query params without change (#408) (734e5b0)
- use Buffer.alloc (#402) (e327108)
- webpack 3 RuleSet creation (#407) (6551c8e)
0.11.0 (2018-07-07)
0.10.2 (2018-07-07)
- plus sign missing (#398) (8885f10)
- support unresolved loader resolve calls (#396) (66086aa)
- use Buffer.alloc (#402) (e327108)
0.10.1 (2018-07-02)
0.10.0 (2018-07-02)
0.9.0 (2018-06-23)
- package: update find-cache-dir to version 2.0.0 (#385) (d871c8a)
- package: update pkg-dir to version 3.0.0 (#384) (d97566d)
0.8.1 (2018-06-14)
0.8.0 (2018-06-14)
0.7.6 (2018-06-12)
- cache: reset corrupted caches (#375) (33cc986)
- thaw SystemImportDeprecationWarning (#379) (5603da0)
- package: add engine to define minimum node version (#377) (d1f9c5f)
0.7.5 (2018-06-04)
- add missing commitlint configuration file (#363) (c878f4b)
- check that normalId is on missingCache (#364) (79aa0bc)
- parse and stringify data in serializers (#358) (c40fd30)
0.7.4 (2018-05-30)
- add missing commitlint configuration file (#363) (c878f4b)
- check that normalId is on missingCache (#364) (79aa0bc)
0.7.3 (2018-05-30)
0.7.2 (2018-05-28)
- missing logMessages import in ParitySystem (#354) (59ec7cf)
- use find-cache-dir for default cache directory (#237) (#355) (0212475)
0.7.1 (2018-05-28)
- Internal reworking splitting out multiple caches and making them less dependent on each other
- Use webpack 4's new tapable plugin interface to avoid the deprecation message on .plugin
- Replace default data serializer, append, with append-2 for performance and stability
- Use brief (chalk) colored log messages
0.7.0-beta.4Replace debug process exit with unfrozen error in source transform (#293)0.7.0-beta.3Monkey patch mini-css CssDependency to update its module's hash (#339)0.7.0-beta.2Fix import context block chunk name support (#327), Add parity compilation cache check0.7.0-beta.1Ensure file and context dependencies are absolute paths (#330), better json error messaging (#307), always chalk log to stderr0.7.0-beta.0Update README0.7.0-alpha.4Add chalk logging with shorter messages, iterate fix on data deleted from md5 cache (#330)0.7.0-alpha.3Update file path naming standards, fix watchRun compiler misnover (#320)0.7.0-alpha.2Fix Source freezing with typescript (#291) (@kristenmills)0.7.0-alpha.1Remove deprecated recordsPath support, more narrow dependency transform generation, add cacache serializer and append-2 serializer as the new default.
- webpack 3 and 4
- node 8 and later
Separate different parts of the cache into subcaches.
- AssetCache: stores the collection of Assets for the compiler
- EnhancedResolverCache: cached results from enhanced-resolver
- Md5Cache: performs file and context hashing together as well as store values
- ModuleCache: stores the modules as frozen and thawed by other plugins
- ModuleResolverCache: stores resolver data collected from enhanced-resolver for a module
Each Cache handles a set of singles to: reset their cache, read from an existing cache, verify the cache is correct, and write out changes to the cache.
The current 'data' serializer, append, is being replaced by append-2. This second version "appends" at the folder level instead of the file level. This make the source easier to understand and write and more likely to hold less bugs. It also happens to be faster than append thanks to being easier to write, it's easier to write optimizations for.
- Thaw dependency instances to their original type
- Naturally thaw NormalModule and ContextModule
- Relate the config hash to the cache directory and frozen items to the compiler context path
- Write items immediately adjacent to each other in AppendSerializer, without filler
0.6.7Add MiniCssExtract support, add graceful failure when module does not freeze all dependencies0.6.6Add local module support and fix webpack 4 json, wasm, and module build hash support0.6.5Fix dynamic import (@faceyspacey, @matthewborden) (#300), fix .split issue (@toriningen) (#289), fix README typo (@Zakjholt, @koningskristof)0.6.4Use general dependency -> block cycle test to avoid max call stack (#283)0.6.3Add safety net to module dependency freezing0.6.2Rebuild activeExports when thawing instead of serializing it0.6.1Fix freezing sourceless modules (#270), fix freezing json and wasm parsers and generators (#276)0.6.0Additional protection against source path being locked when copied0.6.0-alpha.8Protect append serializer from rename and unlink errors on Windows0.6.0-alpha.7Move babel dependencies to development0.6.0-alpha.6Fix unneeded rebuilding performance regressions0.6.0-alpha.5Naturally thaw NormalModule and ContextModule0.6.0-alpha.4Add webpack 4 beta support0.6.0-alpha.3Fix dependency thawing performance regression0.6.0-alpha.2Fix missed preload path relation thawing
Remove the need to depend on webpack record ids feature and to manually invalidate frozen modules due to module id or used harmony export changes.
From hard-source's beginning it froze dependencies as hybrid or abstract dependencies. Thawed they represented the other modules the a frozen module depended on. They made the frozen module's representation of the output brittle. A change in id, used harmony exports, and some other dependency elements meant the frozen module had to thrown away so a new webpack NormalModule could be built and its source rendered. Fixing this debt, the frozen module can re-render as how NormalModule does when a module id or used harmony exports changed.
This improves rebuild time when used exports of a harmony dependency change or webpack assigns a different id to a built module. As well this improves hard-source relability in relation to webpack as it removes the need for the special case logic in hard-source that would invalidates modules in these cases.
Thaw cached NormalModule and ContextModule data into webpack's NormalModule and ContextModule types.
With hybrid dependencies custom module types were needed to be able to use the hybrid dependencies and custom source type. The hybrid modules let webpack trace dependencies through them but left HardSource to pick a lot of work to make sure the custom modules were not returned from webpack hooks if they were out of date. In many cases with webpack 2 and later, to make this work they'd need an additional build in webpack so they the normal process of creating a module would replace a module HardSource returned incorrectly because it could not check that the module was out of date until a later stage in the build.
With this new thawing HardSource is making iterative webpack builds look internally in its data more like the first build that was cached.
Relate cached items to the cache directory and compiler contxt path. A relative cache is a portable cache. The cache built on one computer can be used on another system or in a cloned copy of the project with a different root directory on the same system.
Relate paths in the config hash to the cache directory as the config contains the context path that becomes the compiler context. Two versions of a config may only differ by the context path. Using a different context relative to the root of the project or the cacheDirectory will not be able to use the cached items in the matching config hash's build because the items when their paths are made absolute during the build will be not correct.
Relate frozen items to the compiler context. When thawed the items will have their paths made absolute again so their internal state matches their previous state where paths are absolute during webpack's runtime.
Make resolver request fields relative to the context if they are absolute paths. Most path fields in frozen items can be known ahead of time to be absolute paths. So making the paths relative and absolute while freezing and thawing is straight forward. The request part of resolutions is not straight forward. Sometimes the request path is relative, and sometimes it is absolute. An absolute path cannot be frozen so the absolute path must be made relative. But an original relative request must not be made absolute when thawed. To answer this, make only absolute paths relative and leave the request path as is when thawed. To use these thawed resolutions two checks may be made into the cache. First an unmodified lookup and second a modified lookup with the input made relative as if it was being frozen. This way enhanced-resolve cached items and NormalModule and ContextModule resolver items can be frozen and thawed reliably.
Log new values to byte positions instead of block positions. Removing the filler space in incomplete blocks, this decreases the space AppendSerializer uses and can have a small improvement to write and read performance.
- Internal plugins for freezing and thawing modules, dependencies, and assets
- Use AppendSerializer by default
- Improved ES2015 support
- Environment hash defaults to package-lock.json and yarn.lock if they exist
0.5.16Fix baseMap redundancy logic (#246), fix options passing for cheap-eval devtool (#241)0.5.15Safety check thatsourceMap.mapis in cache before use (@abogaart)0.5.14Improve AppendSerializer performance reading and writing0.5.13Improve AppendSerializer performance on lots of writes0.5.12Fix sourcesContext is undefined (@piecyk)0.5.11Fix relative context module's identifier and id assignment0.5.10Fix contextTimstamp typo and hash regression, fix bad compilerContext when logging freezing errors (#230)0.5.9Fix context dependency validation (#182) and worker-loader cache prefix search (#201)0.5.8- Fix memory usage related to HarmonyExportImportedSpecifierDependency
- Reduce memory usage in frozen modules and source maps
- Reduce memory usage writing to and reading from disk
- Fix using preloaded modules in the memory cache
0.5.7Fix a large memory usage related to HarmonyExportImportedSpecifierDependency (#205)0.5.6Add module freeze error logging0.5.5Fix contextStampm, return context object in synchronous branch0.5.4Improve cache read and write performance related to promises, fix AppendSerializer compacting0.5.3Improve AppendSerializer performance0.5.2Fix nested AsyncDependenciesBlock origins for Stats support
The first step to providing meaningful third party hard-source plugins for deeper integration with advanced webpack plugins, this version adds internally used plugins for freezing and thawing modules, etc. These plugins are accessible externally but are prefixed with -- to help indicate that you should avoid using them in the meantime. You are welcome to make plugins that use these hooks but the interface may change. Internal and external plugin interface changes will only occur during minor version releases before version 1.
As an important example of the utility of the plugin interface modules that are put into a ConcatenatedModule are now cached, helping decreasing build times using that plugin.
- Low level resolution cache
- Missing resolution attempt check
- Expose a 'hard-source-cache-factory' plugin hook on the compiler
- Cache module resolutions in a cache serializer
- Expose a 'hard-source-log' plugin hook on the compiler
- Default options
0.4.15Deserialize objects from their caches before use (@ettavolt)0.4.14Publish index.js and lib, omitting tests0.4.13Add AppendSerializer0.4.12Fix harmony module export * from ... (@filipesilva)0.4.11Fix ContextDependency non-regexp use and fix child compiler cache prefix search0.4.10Bump webpack version to help with nested dependency issues (@rowan)0.4.9Fetch loader options for rule based options in webpack > 20.4.8Fix error in loader rule optimization use during watch mode0.4.7Support webpack 3's loader rules optimization0.4.6Support direct use of SourceMapDevToolPlugin0.4.5Add webpack 3 support. Add initial ModuleConcatenationPlugin support0.4.4Fix configHash default value0.4.3Fix #105. Cache assets in memory as well as disk0.4.2Tune default options0.4.1Add Appveyor CI
Prior versions of hard-source cache resolution values the NormalModuleFactory creates and a similar data for the ContextModuleFactory. These higher level resolutions provide a lot of hard-sources performance gain by reusing those values when resolving modules as long as an assumption holds. The file or context the resolution points at must still exist. The low level resolution cache takes this further caching the work resolving files, loaders, and contexts. These cached resolutions can be reused between multiple module resolutions as they may share files or loaders. Two modules with the same file but different loaders go through separate NormalModuleFactory resolutions meaning they hit the lower level resolvers. This new cache provides some performance gains in regards to the reused values.
Highly related to the low level resolution cache is the missing resolution attempt check that can be made. When the low level resolvers try to find the resource for a request it can optionally build an array of missing paths that it tried. Caching that information hard-source now provides a stronger assurance that what should have been built is. Seeing any of those missing attempts now existing means the old resolution is invalid and any related NormalModuleFactory resolution is also invalid. During the new build, hard-source will let the normal resolutions occur and store the new information.
Leading up to changing the default cache serializer (#53), hard-source has its first plugin hook to make it more flexible. The 'hard-source-cache-factory' plugin hook on the webpack compiler lets a users determine how the cache is write to and read from disk. Documentation on this is in the lib/cache-serializer-factory.js module.
With this hook a working additional serializer and plugin is available to replace the leveldb default serializer with a json serializer. This serializer is primarily available for debugging the contents of the cache as they're directly human readable.
To use this plugin you can add it to your config
plugins: [
// other plugins
new HardSourceWebpackPlugin.HardSourceJsonSerializerPlugin(),As a step to #53 a patch version with a replacement to the leveldb will come out during v0.4.x that will become the default in v0.5.0.
Up until this version the NormalModuleFactory resolutions were using some old code to write and read its cache. That has been replaced with a cache serializer like the other caches. This has little effect on reading but provides a small performance gain when writing changes to the cache from the build. The cache serializers are able to write out changes instead of needing to write out the whole cache. This may a small measurable impact on large project where they were writing out the whole module resolution cache.
A second plugin hook in this release, 'hard-source-log' presents a way to control the logging output from hard-source. Two obvious uses is a plugin to silence its output or another to write the output to disk. Documentation on this plugin hook can be found in lib/logger-factory.js.
With a plugin approach to logging, more logging of lower levels (debug, log) will be added that a plugin will optionally enable for writting out. This additional logging will cover when and why modules are invalidated, timing and other information to help debug hard-source.
Past versions required at least the cacheDirectory and recordsPath options, along with recommending the configHash option. This release sets defaults for these making hard-source easier to use.
cacheDirectorydefaults to'node_modules/.cache/hard-source/[confighash]'recordsPathdefaults to'node_modules/.cache/hard-source/[confighash]/records.json'configHashdefaults torequire('node-object-hash')({sort: false}).hash
- Internal env-hash implementation relying on file hashes
- Support context dependencies and ContextModule
- Preload the memory cache for faster builds
The env-hash implementation is more generalized than the npm env-hash implementation that was previously used. That one focused on hashing modified time values of folders and files to be as fast as possible. This new one is a little slower hashing files and files of folders but supports use cases like reusing a cache from a prior build on a CI environment. While the CI environment would have all new modified times, if the files' content didn't change the hash of those files will be the same letting the old cache be used.
Context dependencies and ContextModules are now supported. This information is stored in the cache and deserialized like file dependencies and NormalModules. This will allow projects with ContextModules and NormalModules with context dependencies to gain the caching benefit they were previously missing.
Webpack's memory cache allows webpack to make assumptions about previously built modules and skip build steps it would otherwise perform on those modules. That cache is now filled with up to date modules from the HardSource cache letting builds with webpack or the first build with a webpack server to gain the performance those assumptions support.
0.3.13Support change to error rendering in webpack 20.3.12Support different resolutions in different child compilations0.3.11Support optional dependencies0.3.10Fix webpack typo'd dependency (HarmonyCompatiblilityDependency)0.3.9Support webpack.IgnorePlugin0.3.8Support webpack 2.2.0 stable release0.3.7Support webpack 2.2.0-rc.4 (by @swernerx)0.3.6Allow webpack 2 rc versions in package peer dependencies0.3.5Support out of order harmony import specifiers0.3.4Support webpack 2.2.0-rc.00.3.3Support webpack 2.1.0-beta.280.3.1Fix false positive invalidation against modules like Delegated and External
Builds and stores checksums of files built by webpack to better determine when to rebuild modules. This helps HardSource rebuild less in CI environments.
Code contributed by:
- @nikhilmat
0.2.7Fix resolve cache invalidation, use resource instead of userRequest0.2.6Don't log version mismatch message on first build without a cache0.2.5Stamp cache with library version and fix serializinglocstrings0.2.4Fix out of date harmony module tree shaking keys0.2.3Fix context depending modules rebuild check0.2.2Flatten dependency loc data for HarmonyImportSpecifier0.2.1Support functions as option to environmentHash
HardSource grew through its 0.0.X versions up to this point. It currently supports:
- Webpack 1 AMD and CommonJS modules
- Webpack 2 Harmony modules with tree shaking
- Module warnings and errors
- Child Compiler use with plugins like ExtractTextWebpackPlugin and HtmlWebpackPlugin
- Webpack builtin plugins like DllPlugin and UglifyJSPlugin
- Module cache busting when dependencies move
- Multiple caches through the configHash option
- Full cache invalidation through dependency hash comparison
0.1.4Consider AMDDefineDependencies as a HardNullDependency like ConstDependency0.1.3Fix normal module resolver plugin that didn't pass on resolve errors0.1.2Ignore ExtractText root modules so that child compilers always run and output assets
0.0.44Prefix module cache identifier by child compiler subcache name0.0.43Flatten dependencylocfield (HMR fix for 0.0.41)0.0.42Support out of order harmony module export specifier dependencies0.0.41Freeze and thaw module warnings and errors (by @Strate)0.0.40Support latest webpack 2 beta (2.1.0-beta.25)0.0.38AddconfigHashoption0.0.37Add a little logic to avoid re-serializing already serialized modules0.0.36Freeze and thaw some common fields that weren't before0.0.35Use additional pass in compilation instead of pre-emptive pass. Fixes use with HMR0.0.34Correctly thaw harmony specifier dependencies0.0.33Add HardModule.libIdent to support DllPlugin0.0.32Fix webpack 2 dependency warnings0.0.30Fix asset thawing and unnecessary dependency invalidation0.0.28Help ensure cacheDirectory is used as users expect0.0.27Use pre-emptive compile when supporting isUsed (webpack 2)0.0.26Invalidate modules based on webpack 2 tree shaking0.0.23Fix to error in resolve invalidation0.0.19Store separate non-devtool related base map for Uglify support0.0.18Invalidate modules depending on a now invalid resolve value0.0.17Support webpack-isomorphic-tools0.0.16Incomplete resolve invalidation0.0.15Invalidate whole cache when environment (node_modules, etc) change0.0.13Freeze and thaw module's_sourceused by Stats0.0.11Store modules in a leveldb store0.0.10Store assets separate for performance0.0.9Support ExtractText and Uglify0.0.8relativecacheDirectorysupport0.0.7Support Harmony modules by disabling tree shaking0.0.5First ReadME, travis badge, cacheDirectory change0.0.4Add source map tests, improve source map thawing0.0.3Add tests