• Create
    • Ask a question
    • Create an article
    • Topics
    • Questions
    • Articles
    • Users
    • Badges
  • Sign in
  • Home /
avatar image
0

Spark Not able to run SQL more than 64KB size

spark-sqlspark 1.6.1
Question by 83sand · Jul 05, 2016 at 03:36 PM ·

I am facing issues in running spark sql (SQL size around 85KB) on a dataset. It seems we can not have an SQL more than 64KB in spark 1.6.1 version.

I do see this issue is being fixed in Spark 2.0. https://issues.apache.org/jira/browse/SPARK-15285

My cluster is still on 1.6.1. Is there a temporary work around to get 85kb SQL run on 1.6.1? Thanks for your help.

Add comment
Comment
10 |600 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Sort

  • Votes
  • Created
  • Oldest

Your answer

Hint: You can notify a user about this post by typing @username

Up to 2 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

Follow this Question

7 People are following this question.

avatar image avatar image avatar image avatar image avatar image avatar image avatar image

Related Questions

How to write data in the dataframe into single .parquet file(both data & metadata in single file) in HDFS? 0 Answers

Error access the map data type column(parquet) in spark-sql 1 Answer

Spark SQL - READ and WRITE in sequence or pipeline? 1 Answer

I have spark-sql query, i would need to save/insert result set into HDFS directory, could some please suggest me the best way? 3 Answers

Is it possible to read a CSV file via SFTP using spark-csv 3 Answers

  • Product
    • Databricks Cloud
    • FAQ
  • Spark
    • About Spark
    • Developer Resources
    • Community + Events
  • Services
    • Certification
    • Spark Support
    • Spark Training
  • Company
    • About Us
    • Team
    • News
    • Contact
  • Careers
  • Blog

Databricks Inc.
160 Spear Street, 13th Floor
San Francisco, CA 94105

info@databricks.com
1-866-330-0121

  • Twitter
  • LinkedIn
  • Facebook
  • Facebook

© Databricks 2015. All rights reserved. Apache Spark and the Apache Spark Logo are trademarks of the Apache Software Foundation.

  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Create an article
  • Explore
  • Topics
  • Questions
  • Articles
  • Users
  • Badges