Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Webpack #9

Merged
merged 19 commits into from
Jan 10, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,6 @@ gh-pages/
node_modules/
components/
bower_components/
/spec/*.js
/spec/*.xml
/spec/*.html
36 changes: 33 additions & 3 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,37 @@
language: node_js
node_js:
- '0.10'
- '6'
before_install:
- wget https://s3-us-west-2.amazonaws.com/cdn.thegrid.io/caliper/libvips/libccv-0.1.1.tar.gz
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nooo, what is this stuff? This is custom C++ libraries. Not cross-compiled for browser either. Not acceptable to include here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The thing is, otherwise we can't install noflo-ccv via NPM, and hence expose its browser components.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be super-happy to not need these native packages, but I wonder how to make NPM skip the deps that need them

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So either

  • Split out the things with custom Node.js C++ dependencies to another library
  • Make the dependencies optional and handle them not existing
  • Drop the offending NoFlo libraries from the build

- mkdir $HOME/libccv
- echo $HOME/libccv
- tar -xvzf libccv-0.1.1.tar.gz -C $HOME/libccv
before_script:
- npm install -g grunt-cli
after_success:
- grunt gh-pages
deploy:
provider: script
skip_cleanup: true
script: grunt gh-pages
on:
tags: true
repo: noflo/noflo-browser
env:
matrix:
- CXX=g++-4.8
global:
secure: mXwYeFfEjRsB31W5sorAjBXNPc2ZqV1IabgGv4fx8nmLGJ2KVCg7FRtJxPkshCxUMbn69S5JR12HcKWsFNY4uF0un7On70ggcq1+TpRFh8VkDr2KuRTsxW0NuZE3SO/xx7fxMQ98woHnW1FqLrQMhlZLDdW+OSVrx8/sJ4Xql3oZgt/n3VGaIQuJ6D9cvx/TT8WqxF+cEunm1T0eHpjtVTEKQHFDsOtztWJR/tHDrxpP0J8OrpEXtdFGXXbS8od4wCSJ0qmZXPfuD7cQgwwMLa3FnHxC5B4SOcjOagr8cr269BTSdV8CN5udVb8g2vuvF33T/Q7KFZKd/G9ue1w+7unyAG6MZT6cN3uzL5JwsAsU9C8fRF94yD7Q3dfkI+UMssf0kt3zfBU/XgvJJQ4mzoaouGcgdB2/jn0PbQrECUfILhG4/c7Y/6hQUjxJelc2hunXkY7QwOuRi4xNGsTgEZwBT7pZmPXRwOABZwdo/4WU6y4vTKINyX4f4di/bIOtewoTT+tYwab9A7xqka8SUTyc1FHuQyzAww9BK4TGjaWxKP7dYPmLdKWzGPUhEKwzsNBIKvB6g5vIULb6T7x4/UXOlI7hKnM1rUOpiYLto4RV4LJlIksUKpaC8z/dtmKko3qThXWNJc9tF6trCEUjV03xXfgAh1ohsV0/euVwYLk=
addons:
apt:
sources:
- ubuntu-toolchain-r-test
packages:
- libpng-dev
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same, all this crack introduced by libccv has to stay away. Cannot legitmately say to people that to build NoFlo you need all this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not only noflo-ccv, but also noflo-image and noflo-canvas

Copy link
Member

@jonnor jonnor Jan 10, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does not change the point. These dependencies are awful and should be completely uneccesary for a browser project. This will mean lots of people on Mac won't be able to build noflo-browser. Basically no-one on Windows will be able. Even some people Linux people will struggle.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is all true. The right approach would be to fix it upstream in the modules with native dependencies:

But until that is done, I don't really see other options than either including these native deps on Travis, or removing those modules from package.json. But we need them for bunch of Flowhub demos...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be noted that normal browser builds for NoFlo don't require these. Only projects that use those three modules.

https://github.com/noflo/noflo-browser-app/blob/master/.travis.yml

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bergie At the very least, we got to file bugs on all the offending libraries so we can fix it later. If NPM is to be our solution for NoFlo on browser then we've got to find solutions for these kinds of issues.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe also have a tracking bug here which ties the general-problem together.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bergie what will be our recommendation to future libraries that need some (native) dependecies for Node.js and others/none for browser?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filed here, #10

- libatlas-base-dev
- libblas-dev
- libcairo2-dev
- libjpeg8-dev
- libpango1.0-dev
- libgif-dev
- graphicsmagick
- build-essential
- g++-4.8
127 changes: 99 additions & 28 deletions Gruntfile.coffee
Original file line number Diff line number Diff line change
Expand Up @@ -6,25 +6,48 @@ module.exports = ->
@initConfig
pkg: @file.readJSON 'package.json'

# Updating the package manifest files
noflo_manifest:
update:
files:
'component.json': ['graphs/*', 'components/*']
'package.json': ['graphs/*', 'components/*']

# Browser build of NoFlo
'bower-install-simple':
deps:
options:
interactive: false
forceLatest: false
directory: 'browser/bower_components'

noflo_browser:
everything:
options:
exposed_modules:
'noflo': 'noflo'
'noflo-runtime-iframe': 'noflo-runtime-iframe'
'noflo-runtime-webrtc': 'noflo-runtime-webrtc'
'fbp-protocol-client': 'fbp-protocol-client'
'noflo-runtime': 'noflo-runtime'
manifest:
runtimes: [
'noflo'
]
discover: true
recursive: true
subdirs: false
debug: true
webpack:
externals:
'repl': 'commonjs repl' # somewhere inside coffee-script
'module': 'commonjs module' # somewhere inside coffee-script
'child_process': 'commonjs child_process' # somewhere inside coffee-script
'jison': 'commonjs jison'
'canvas': 'commonjs canvas'
'mimetype': 'commonjs mimetype'
'should': 'commonjs should' # used by tests in octo
'express': 'commonjs express' # used by tests in octo
'highlight': 'commonjs highlight' # used by octo?
'microflo-emscripten': 'commonjs microflo-emscripten' # optional?
'acorn': 'commonjs acorn' # optional?
module:
loaders: [
{ test: /\.coffee$/, loader: "coffee-loader" }
{ test: /\.json$/, loader: "json-loader" }
{ test: /\.fbp$/, loader: "fbp-loader" }
]
resolve:
extensions: ["", ".coffee", ".js"]
alias:
URIjs: 'urijs'
node:
fs: "empty"
heads: [
"""<style>
body {
Expand All @@ -34,26 +57,40 @@ module.exports = ->
}
</style>"""
,
"""<script src="./bower_components/coffee-script/extras/coffee-script.js"></script>"""
,
"""<script src="../node_modules/babel-core/browser.js"></script>"""
"""<script src="https://cdnjs.cloudflare.com/ajax/libs/coffee-script/1.7.1/coffee-script.min.js"></script>"""
,
"""
<script src="./bower_components/requirejs/require.js"></script>
<script src="vendor/requirejs/require.js"></script>
<script>
requirejs.config({
packages: [
{
name: 'React',
location: './bower_components/react',
location: 'vendor/react',
main: 'react'
}
]
});
</script>"""
]
files:
"browser/everything.js": ['component.json']
"browser/everything.js": ['package.json']

copy:
vendor:
files: [
expand: true
cwd: 'node_modules/react/dist'
src: '*.js'
dest: 'browser/vendor/react'
filter: 'isFile'
,
expand: true
cwd: 'node_modules/requirejs'
src: '*.js'
dest: 'browser/vendor/requirejs'
filter: 'isFile'
]

manifest:
cache:
Expand All @@ -64,7 +101,7 @@ module.exports = ->
dest: 'browser/manifest.appcache'
src: [
'everything.*'
'bower_components/*/*'
'vendor/*/**.js'
]

'string-replace':
Expand All @@ -74,14 +111,43 @@ module.exports = ->
options:
replacements: [
pattern: '<html>'
replacement: '<html manifest="/noflo-browser/manifest.appcache">'
replacement: '<html manifest="manifest.appcache">'
]

# CoffeeScript compilation
coffee:
spec:
options:
bare: true
expand: true
cwd: 'spec'
src: ['**.coffee']
dest: 'spec'
ext: '.js'
# Web server for the browser tests
connect:
server:
options:
port: 8000
# BDD tests on browser
noflo_browser_mocha:
all:
options:
scripts: ["../browser/everything.js"]
files:
'spec/runner.html': ['spec/*.js']
mocha_phantomjs:
all:
options:
output: 'spec/result.xml'
reporter: 'spec'
urls: ['http://localhost:8000/spec/runner.html']
failWithOutput: true
'gh-pages':
options:
base: 'browser'
clone: 'gh-pages'
message: 'Updating'
message: "Release #{pkg.name} #{process.env.TRAVIS_TAG}"
repo: repo
user:
name: 'NoFlo bot'
Expand All @@ -90,26 +156,31 @@ module.exports = ->
src: '**/*'

# Grunt plugins used for building
@loadNpmTasks 'grunt-bower-install-simple'
@loadNpmTasks 'grunt-noflo-manifest'
@loadNpmTasks 'grunt-noflo-browser'
@loadNpmTasks 'grunt-contrib-copy'
@loadNpmTasks 'grunt-manifest'
@loadNpmTasks 'grunt-string-replace'

# Grunt plugins used for testing
@loadNpmTasks 'grunt-contrib-coffee'
@loadNpmTasks 'grunt-contrib-connect'
@loadNpmTasks 'grunt-mocha-phantomjs'

# Grunt plugins used for deploying
@loadNpmTasks 'grunt-gh-pages'

# Our local tasks
@registerTask 'build', 'Build NoFlo for the chosen target platform', (target = 'all') =>
@task.run 'bower-install-simple'
@task.run 'noflo_manifest'
@task.run 'noflo_browser'
@task.run 'copy'
@task.run 'manifest'
@task.run 'string-replace:manifest'

@registerTask 'test', 'Build NoFlo and run automated tests', (target = 'all') =>
@task.run 'build'
@task.run 'coffee'
@task.run 'connect'
@task.run 'noflo_browser_mocha'
@task.run 'mocha_phantomjs'

@registerTask 'default', ['test']
22 changes: 0 additions & 22 deletions bower.json

This file was deleted.

63 changes: 0 additions & 63 deletions component.json

This file was deleted.

Loading