Skip to content

Instantly share code, notes, and snippets.

@codspire
Last active December 2, 2021 03:54
Show Gist options
  • Select an option

  • Save codspire/7b0955b9e67fe73f6118dad9539cbaa2 to your computer and use it in GitHub Desktop.

Select an option

Save codspire/7b0955b9e67fe73f6118dad9539cbaa2 to your computer and use it in GitHub Desktop.
Making Zeppelin, Spark, pyspark work on Windows
@reflog
Copy link
Copy Markdown

reflog commented Sep 9, 2017

thanks! this was very helpful, I was wrecking my head trying to make zeppelin to start, and your approach worked!

@mravikrishna
Copy link
Copy Markdown

Thanks for sharing. Worked for me

@jflight1
Copy link
Copy Markdown

Worked - thanks!!!

@IshwarBhat
Copy link
Copy Markdown

This is a lifesaver!

@allixender
Copy link
Copy Markdown

+1 very strange behaviour, but now works 😄

@evandroc
Copy link
Copy Markdown

It works here too, but I had to make a change on the process.
After the command

$ cp %SPARK_HOME%\jars*.jar %ZEPPELIN_HOME%\interpreter\spark

I also had to remove the folder %ZEPPELIN_HOME%\interpreter\spark\pyspark

@skd93
Copy link
Copy Markdown

skd93 commented Feb 14, 2018

Hi,

When you say this, Rename %ZEPPELIN_HOME%\conf\zeppelin-env.cmd.template to %ZEPPELIN_HOME%\conf\zeppelin-env.cmd

Do you mean converting the template file to .cmd file? If yes how can i modify it and add additional texts to it?

@mcdoyaji
Copy link
Copy Markdown

@skd93 don't need any converting. just remove .template part from the filename for run .cmd on windows console.

@mcdoyaji
Copy link
Copy Markdown

mcdoyaji commented Jun 12, 2018

@ar1272
Copy link
Copy Markdown

ar1272 commented Jul 24, 2018

Thanks a bunch!!! This helped a lot!

@juliovg
Copy link
Copy Markdown

juliovg commented Sep 4, 2018

hello , i follow all your steps and i get the error:

error_zeppelin

can you help me ?

@kiran2531159
Copy link
Copy Markdown

try installing Zeppelin version - zeppelin-0.7.2-bin-all with all interpreters

@vishrtd
Copy link
Copy Markdown

vishrtd commented Sep 13, 2018

hello , i follow all your steps and i get the error:

error_zeppelin

can you help me ?

Where have you added the line
set SPARK_HOME=

I faced a similar issue, :-p you have to add it below:

:MAIN
call "%bin%\common.cmd"

See if this helps!

@lbcoker
Copy link
Copy Markdown

lbcoker commented Oct 15, 2018

I setup as above but still getting issue with zeppelin. Anyone ever seen this?

c:\tools\zeppelin-0.7.3>bin\zeppelin.cmd start
The system cannot find the path specified.

No idea what it is not finding.

@nimmmalarohit
Copy link
Copy Markdown

I setup as above but still getting issue with zeppelin. Anyone ever seen this?

c:\tools\zeppelin-0.7.3>bin\zeppelin.cmd start
The system cannot find the path specified.

No idea what it is not finding.

You must have set the environment variables incorrectly, remove zepellin_home from environment variables and check once.

@shatestest
Copy link
Copy Markdown

@shatestest
Copy link
Copy Markdown

@baram204 I am trying to install zeppelin-0.8.0 version on windows 8 r2 getting below error , any help please
https://stackoverflow.com/questions/54312233/zeppeling-throwing-nullpointerexception-while-configuring

I am working scala with spark 2.3.1 , do I still need pyspark ? Why do I need ?

@samsebk
Copy link
Copy Markdown

samsebk commented Feb 3, 2019

thank you. Your method works!!!
Also, the recommended Zeppelin version zeppelin-0.7.2-bin-all works, but those versions after 0.7.2. I think there is some error in the Common.cmd

Really appreciate your help.

@mxku
Copy link
Copy Markdown

mxku commented Mar 13, 2019

yeah. in 0.8.1 common.cmd is broken.

You can fix this by removing curly brace '{' from below part

if defined ZEPPELIN_JMX_ENABLE ( if not defined ZEPPELIN_JMX_PORT ( set ZEPPELIN_JMX_PORT="9996" ) set JMX_JAVA_OPTS=" -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=%ZEPPELIN_JMX_PORT% -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false" set ZEPPELIN_JAVA_OPTS=%JMX_JAVA_OPTS% %ZEPPELIN_JAVA_OPTS% )

At another place a variable is replaced in sh format ${variable.name} change that to cmd format %variable.name%

@darealagung
Copy link
Copy Markdown

HI,
I already successfully installed and run hadoop, spark, pyspark, jupyter notebook. But with zeppelin 0.8, I am just stuck. Typing zeppelin.cmd gave nothing, immediately goes to prompt again with no error, no nothing. I tried your solution here, with same result. Can somebody help me? Thanks,

@msid
Copy link
Copy Markdown

msid commented Aug 27, 2019

HI,
I already successfully installed and run hadoop, spark, pyspark, jupyter notebook. But with zeppelin 0.8, I am just stuck. Typing zeppelin.cmd gave nothing, immediately goes to prompt again with no error, no nothing. I tried your solution here, with same result. Can somebody help me? Thanks,

Had the same issue with 0.8.1. Fixed it with replacing curly brace '}' with ')' in bin/common.cmd #77 (a little bit different from what manishonline wrote)

...
if defined ZEPPELIN_JMX_ENABLE (
    if not defined ZEPPELIN_JMX_PORT (
        set ZEPPELIN_JMX_PORT="9996"
    }
...

@darealagung
Copy link
Copy Markdown

darealagung commented Aug 27, 2019 via email

@tejaswishetty17
Copy link
Copy Markdown

tejaswishetty17 commented Sep 12, 2019

zeppelin_error2
I followed all the installation steps, but the zeppelin server fails to start. Please check the attached image.

@darealagung
Copy link
Copy Markdown

darealagung commented Sep 12, 2019 via email

@tejaswishetty17
Copy link
Copy Markdown

tejaswishetty17 commented Sep 12, 2019 via email

@gitgmontoya
Copy link
Copy Markdown

You are the savior!!!

@truthelectron
Copy link
Copy Markdown

nice hack tried it on the zeppelin 0.8.2. could only get the spark to work. but then python did not work. this is a great notebook don't know why it really difficult to get it up and running added to this difficulty there are no proper tutorials also.

@zhongxur
Copy link
Copy Markdown

Very appreciated for making this notebook! Successfully get Spark work but pyspark/python not working. it reports code = compile(mod, '', 'exec')
TypeError: required field "type_ignores" missing from Module
Thanks again for sharing this notebook!

@zeinramadan
Copy link
Copy Markdown

I'm getting this error: TypeError: namedtuple() missing 3 required keyword-only arguments: 'verbose', 'rename', and 'module'
when executing the pyspark code in zeppelin, any ideas?

@zeinramadan
Copy link
Copy Markdown

If anyone encounters this issue, replace line 393 in serializers.py with this line:
cls = _old_namedtuple(*args, **kwargs, verbose=False, rename=False, module=None)
Please check this stackoverflow post about this: https://stackoverflow.com/a/42615678/9691413

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment