![]() ![]() You can use two DataFrameReader APIs to specify partitioning: See the Spark SQL programming guide for other parameters, such as fetchsize, that can help with performance. Each task is spread across the executors, which can increase the parallelism of the reads and writes through the JDBC interface. In the Spark UI, you can see that the number of partitions dictate the number of tasks that are launched.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |