Skip to content

Multiple fixes and improvements#1

Open
felipecsl wants to merge 8 commits into
mrflip:masterfrom
felipecsl:master
Open

Multiple fixes and improvements#1
felipecsl wants to merge 8 commits into
mrflip:masterfrom
felipecsl:master

Conversation

@felipecsl

Copy link
Copy Markdown
  • Fix gemspec files and dependency on extlib

This fixes 3 issues when installing/using the gem:

1 Missing files referenced in gemspec:

wukong at /Users/felipe/.rvm/gems/ruby-1.9.3-p194/bundler/gems/wukong-d656650bdae9 did not have a valid gemspec.
This prevents bundler from installing bins or native extensions, but that may not affect its functionality.
The validation message from Rubygems was:
  ["examples/server_logs/breadcrumb_edges.rb", "examples/server_logs/histograms.rb", "examples/server_logs/page_counts.pig", "notes/away/icss/icss_specification.textile", "notes/away/swineherd/swineherd-old-readme.textile", "notes/away/wukong-fs/wukong-fs-old-readme.textile", "notes/away/wukong-transform/wukong-processor-old-readme.textile", "spec/data/a_atsigns_b.tsv", "spec/data/a_follows_b.tsv", "spec/data/tweet.tsv", "spec/data/twitter_user.tsv"] are not files

2 Missing dependency on extlib:

➜  wukong git:(master) ✗ bundle exec irb

1.9.3-p194 :001 > require 'wukong'

LoadError: cannot load such file -- extlib/blank

3 Missing dep on right_aws:

/Users/felipe/.rvm/gems/ruby-1.9.3-p194/bundler/gems/wukong-f25cbb17dbd1/lib/wukong/script/emr_command.rb:1:in `require': cannot load such file -- right_aws (LoadError)
    from /Users/felipe/.rvm/gems/ruby-1.9.3-p194/bundler/gems/wukong-f25cbb17dbd1/lib/wukong/script/emr_command.rb:1:in `<top (required)>'
    from /Users/felipe/.rvm/gems/ruby-1.9.3-p194/bundler/gems/wukong-f25cbb17dbd1/lib/wukong/script.rb:154:in `require'
    from /Users/felipe/.rvm/gems/ruby-1.9.3-p194/bundler/gems/wukong-f25cbb17dbd1/lib/wukong/script.rb:154:in `run'
    from /Users/felipe/.rvm/gems/ruby-1.9.3-p194/bundler/gems/wukong-f25cbb17dbd1/lib/wukong.rb:16:in `run'

@felipecsl

Copy link
Copy Markdown
Author

Also found and fixed a problem with emr_extra_args where it was exploding here:

/Users/felipe/Data/Projects/wukong/lib/wukong/script/emr_command.rb:76:in `execute_emr_runner': can't convert String into Array (TypeError)
    from /Users/felipe/Data/Projects/wukong/lib/wukong/script/emr_command.rb:32:in `execute_emr_workflow'
    from /Users/felipe/Data/Projects/wukong/lib/wukong/script.rb:155:in `run'
    from /Users/felipe/Data/Projects/wukong/lib/wukong.rb:16:in `run'
    from visits_runner.rb:7:in `<main>'

@felipecsl

Copy link
Copy Markdown
Author

Added Settings config option to allow specifying which AWS region to run the job via emr_region option

@mrflip

mrflip commented Oct 4, 2012

Copy link
Copy Markdown
Owner

sorry for neglecting this -- we've moved the primary wukong branch to infochimps-labs/wukong, so I totally missed this. If I haven't merged this in by later this week please poke us: http://github.com/infochimps-labs/wukong/issues.

@felipecsl

Copy link
Copy Markdown
Author

Alright @mrflip thanks :)

@felipecsl

Copy link
Copy Markdown
Author

Any update on this, @mrflip ?

@felipecsl

Copy link
Copy Markdown
Author

Is this repository still maintained, or we should be using the infochimps-labs version?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants