21/03/16 21:30:35 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
21/03/16 21:30:36 INFO SparkContext: Running Spark version 2.4.3
21/03/16 21:30:36 INFO SparkContext: Submitted application: sparklyr
21/03/16 21:30:36 INFO SecurityManager: Changing view acls to: nathan
21/03/16 21:30:36 INFO SecurityManager: Changing modify acls to: nathan
21/03/16 21:30:36 INFO SecurityManager: Changing view acls groups to: 
21/03/16 21:30:36 INFO SecurityManager: Changing modify acls groups to: 
21/03/16 21:30:36 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(nathan); groups with view permissions: Set(); users  with modify permissions: Set(nathan); groups with modify permissions: Set()
21/03/16 21:30:36 INFO Utils: Successfully started service 'sparkDriver' on port 49382.
21/03/16 21:30:36 INFO SparkEnv: Registering MapOutputTracker
21/03/16 21:30:36 INFO SparkEnv: Registering BlockManagerMaster
21/03/16 21:30:36 INFO BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
21/03/16 21:30:36 INFO BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
21/03/16 21:30:36 INFO DiskBlockManager: Created local directory at /private/var/folders/x3/lznkklbd17n2ljvwmyg8g1vc0000gn/T/blockmgr-5f368db3-72a1-4877-8110-3e4341b44c30
21/03/16 21:30:36 INFO MemoryStore: MemoryStore started with capacity 912.3 MB
21/03/16 21:30:36 INFO SparkEnv: Registering OutputCommitCoordinator
21/03/16 21:30:37 INFO Utils: Successfully started service 'SparkUI' on port 4040.
21/03/16 21:30:37 INFO SparkUI: Bound SparkUI to 127.0.0.1, and started at http://localhost:4040
21/03/16 21:30:37 INFO SparkContext: Added JAR file:/Library/Frameworks/R.framework/Versions/4.0/Resources/library/sparklyr/java/sparklyr-2.4-2.11.jar at spark://localhost:49382/jars/sparklyr-2.4-2.11.jar with timestamp 1615926637358
21/03/16 21:30:37 INFO Executor: Starting executor ID driver on host localhost
21/03/16 21:30:37 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 49383.
21/03/16 21:30:37 INFO NettyBlockTransferService: Server created on localhost:49383
21/03/16 21:30:37 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
21/03/16 21:30:37 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, localhost, 49383, None)
21/03/16 21:30:37 INFO BlockManagerMasterEndpoint: Registering block manager localhost:49383 with 912.3 MB RAM, BlockManagerId(driver, localhost, 49383, None)
21/03/16 21:30:37 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, localhost, 49383, None)
21/03/16 21:30:37 INFO BlockManager: Initialized BlockManager: BlockManagerId(driver, localhost, 49383, None)
21/03/16 21:30:38 INFO SharedState: loading hive config file: file:/Users/nathan/spark/spark-2.4.3-bin-hadoop2.7/conf/hive-site.xml
21/03/16 21:30:38 INFO SharedState: Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir ('file:/Users/nathan/Documents/work/NE%20Data%20Ltd/projects/catalog/inst/tinytest/spark-warehouse').
21/03/16 21:30:38 INFO SharedState: Warehouse path is 'file:/Users/nathan/Documents/work/NE%20Data%20Ltd/projects/catalog/inst/tinytest/spark-warehouse'.
21/03/16 21:30:39 INFO StateStoreCoordinatorRef: Registered StateStoreCoordinator endpoint
21/03/16 21:30:41 INFO HiveUtils: Initializing HiveMetastoreConnection version 1.2.1 using Spark classes.
21/03/16 21:30:42 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
21/03/16 21:30:42 INFO ObjectStore: ObjectStore, initialize called
21/03/16 21:30:42 INFO Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
21/03/16 21:30:42 INFO Persistence: Property datanucleus.cache.level2 unknown - will be ignored
21/03/16 21:30:44 INFO ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
21/03/16 21:30:45 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
21/03/16 21:30:45 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
21/03/16 21:30:46 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
21/03/16 21:30:46 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
21/03/16 21:30:46 INFO MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
21/03/16 21:30:46 INFO ObjectStore: Initialized ObjectStore
21/03/16 21:30:46 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.0
21/03/16 21:30:47 WARN ObjectStore: Failed to get database default, returning NoSuchObjectException
21/03/16 21:30:47 INFO HiveMetaStore: Added admin role in metastore
21/03/16 21:30:47 INFO HiveMetaStore: Added public role in metastore
21/03/16 21:30:47 INFO HiveMetaStore: No user is added in admin role, since config is empty
21/03/16 21:30:47 INFO HiveMetaStore: 0: get_all_databases
21/03/16 21:30:47 INFO audit: ugi=nathan	ip=unknown-ip-addr	cmd=get_all_databases	
21/03/16 21:30:47 INFO HiveMetaStore: 0: get_functions: db=default pat=*
21/03/16 21:30:47 INFO audit: ugi=nathan	ip=unknown-ip-addr	cmd=get_functions: db=default pat=*	
21/03/16 21:30:47 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MResourceUri" is tagged as "embedded-only" so does not have its own datastore table.
21/03/16 21:30:47 INFO SessionState: Created local directory: /var/folders/x3/lznkklbd17n2ljvwmyg8g1vc0000gn/T/31fcc77a-b36b-4f71-80f6-a5310d2ff88f_resources
21/03/16 21:30:47 INFO SessionState: Created HDFS directory: /tmp/hive/nathan/31fcc77a-b36b-4f71-80f6-a5310d2ff88f
21/03/16 21:30:47 INFO SessionState: Created local directory: /var/folders/x3/lznkklbd17n2ljvwmyg8g1vc0000gn/T/nathan/31fcc77a-b36b-4f71-80f6-a5310d2ff88f
21/03/16 21:30:47 INFO SessionState: Created HDFS directory: /tmp/hive/nathan/31fcc77a-b36b-4f71-80f6-a5310d2ff88f/_tmp_space.db
21/03/16 21:30:47 INFO HiveClientImpl: Warehouse location for Hive client (version 1.2.2) is file:/Users/nathan/Documents/work/NE%20Data%20Ltd/projects/catalog/inst/tinytest/spark-warehouse
21/03/16 21:30:47 INFO HiveMetaStore: 0: get_database: default
21/03/16 21:30:47 INFO audit: ugi=nathan	ip=unknown-ip-addr	cmd=get_database: default	
21/03/16 21:30:47 INFO HiveMetaStore: 0: get_table : db=default tbl=tab1
21/03/16 21:30:47 INFO audit: ugi=nathan	ip=unknown-ip-addr	cmd=get_table : db=default tbl=tab1	
21/03/16 21:30:47 INFO HiveMetaStore: 0: get_database: default
21/03/16 21:30:47 INFO audit: ugi=nathan	ip=unknown-ip-addr	cmd=get_database: default	
21/03/16 21:30:47 INFO HiveMetaStore: 0: get_table : db=default tbl=tab1
21/03/16 21:30:47 INFO audit: ugi=nathan	ip=unknown-ip-addr	cmd=get_table : db=default tbl=tab1	
21/03/16 21:30:47 INFO HiveMetaStore: 0: get_database: default
21/03/16 21:30:47 INFO audit: ugi=nathan	ip=unknown-ip-addr	cmd=get_database: default	
21/03/16 21:30:47 INFO HiveMetaStore: 0: get_table : db=default tbl=tab1
21/03/16 21:30:47 INFO audit: ugi=nathan	ip=unknown-ip-addr	cmd=get_table : db=default tbl=tab1	
21/03/16 21:30:48 INFO HiveExternalCatalog: Persisting file based data source table `default`.`tab1` into Hive metastore in Hive compatible format.
21/03/16 21:30:48 INFO HiveMetaStore: 0: create_table: Table(tableName:tab1, dbName:default, owner:nathan, createTime:1615926641, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:fieldOne, type:bigint, comment:null)], location:file:/Users/nathan/Documents/work/NE Data Ltd/projects/catalog/inst/tinytest/, /var/folders/x3/lznkklbd17n2ljvwmyg8g1vc0000gn/T/RtmpL2S7t7/file24c544fe245b, inputFormat:org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe, parameters:{path=file:/Users/nathan/Documents/work/NE Data Ltd/projects/catalog/inst/tinytest/, /var/folders/x3/lznkklbd17n2ljvwmyg8g1vc0000gn/T/RtmpL2S7t7/file24c544fe245b, serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{})), partitionKeys:[FieldSchema(name:partCol, type:int, comment:null)], parameters:{EXTERNAL=TRUE, spark.sql.sources.schema.numPartCols=1, spark.sql.sources.schema.part.0={"type":"struct","fields":[{"name":"fieldOne","type":"long","nullable":true,"metadata":{}},{"name":"partCol","type":"integer","nullable":true,"metadata":{}}]}, spark.sql.sources.schema.partCol.0=partCol, spark.sql.partitionProvider=catalog, spark.sql.sources.schema.numParts=1, spark.sql.sources.provider=parquet, spark.sql.create.version=2.4.3}, viewOriginalText:null, viewExpandedText:null, tableType:EXTERNAL_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{}, groupPrivileges:null, rolePrivileges:null))
21/03/16 21:30:48 INFO audit: ugi=nathan	ip=unknown-ip-addr	cmd=create_table: Table(tableName:tab1, dbName:default, owner:nathan, createTime:1615926641, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:fieldOne, type:bigint, comment:null)], location:file:/Users/nathan/Documents/work/NE Data Ltd/projects/catalog/inst/tinytest/, /var/folders/x3/lznkklbd17n2ljvwmyg8g1vc0000gn/T/RtmpL2S7t7/file24c544fe245b, inputFormat:org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe, parameters:{path=file:/Users/nathan/Documents/work/NE Data Ltd/projects/catalog/inst/tinytest/, /var/folders/x3/lznkklbd17n2ljvwmyg8g1vc0000gn/T/RtmpL2S7t7/file24c544fe245b, serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{})), partitionKeys:[FieldSchema(name:partCol, type:int, comment:null)], parameters:{EXTERNAL=TRUE, spark.sql.sources.schema.numPartCols=1, spark.sql.sources.schema.part.0={"type":"struct","fields":[{"name":"fieldOne","type":"long","nullable":true,"metadata":{}},{"name":"partCol","type":"integer","nullable":true,"metadata":{}}]}, spark.sql.sources.schema.partCol.0=partCol, spark.sql.partitionProvider=catalog, spark.sql.sources.schema.numParts=1, spark.sql.sources.provider=parquet, spark.sql.create.version=2.4.3}, viewOriginalText:null, viewExpandedText:null, tableType:EXTERNAL_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{}, groupPrivileges:null, rolePrivileges:null))	
21/03/16 21:30:48 INFO FileUtils: Creating directory if it doesn't exist: file:/Users/nathan/Documents/work/NE Data Ltd/projects/catalog/inst/tinytest/, /var/folders/x3/lznkklbd17n2ljvwmyg8g1vc0000gn/T/RtmpL2S7t7/file24c544fe245b
21/03/16 21:30:48 INFO HiveMetaStore: 0: get_database: global_temp
21/03/16 21:30:48 INFO audit: ugi=nathan	ip=unknown-ip-addr	cmd=get_database: global_temp	
21/03/16 21:30:48 WARN ObjectStore: Failed to get database global_temp, returning NoSuchObjectException
21/03/16 21:30:49 INFO CodeGenerator: Code generated in 364.783112 ms
21/03/16 21:30:49 INFO ContextCleaner: Cleaned accumulator 1
21/03/16 21:30:50 INFO CodeGenerator: Code generated in 49.246776 ms
21/03/16 21:30:50 INFO CodeGenerator: Code generated in 16.839148 ms
21/03/16 21:30:50 INFO SparkContext: Starting job: count at utils.scala:135
21/03/16 21:30:50 INFO DAGScheduler: Registering RDD 3 (count at utils.scala:135)
21/03/16 21:30:50 INFO DAGScheduler: Got job 0 (count at utils.scala:135) with 1 output partitions
21/03/16 21:30:50 INFO DAGScheduler: Final stage: ResultStage 1 (count at utils.scala:135)
21/03/16 21:30:50 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 0)
21/03/16 21:30:50 INFO DAGScheduler: Missing parents: List(ShuffleMapStage 0)
21/03/16 21:30:50 INFO DAGScheduler: Submitting ShuffleMapStage 0 (MapPartitionsRDD[3] at count at utils.scala:135), which has no missing parents
21/03/16 21:30:50 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 7.9 KB, free 912.3 MB)
21/03/16 21:30:50 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 4.2 KB, free 912.3 MB)
21/03/16 21:30:50 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on localhost:49383 (size: 4.2 KB, free: 912.3 MB)
21/03/16 21:30:50 INFO SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:1161
21/03/16 21:30:50 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 0 (MapPartitionsRDD[3] at count at utils.scala:135) (first 15 tasks are for partitions Vector(0))
21/03/16 21:30:50 INFO TaskSchedulerImpl: Adding task set 0.0 with 1 tasks
21/03/16 21:30:50 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, localhost, executor driver, partition 0, PROCESS_LOCAL, 7882 bytes)
21/03/16 21:30:50 INFO Executor: Running task 0.0 in stage 0.0 (TID 0)
21/03/16 21:30:50 INFO Executor: Fetching spark://localhost:49382/jars/sparklyr-2.4-2.11.jar with timestamp 1615926637358
21/03/16 21:30:50 INFO TransportClientFactory: Successfully created connection to localhost/127.0.0.1:49382 after 22 ms (0 ms spent in bootstraps)
21/03/16 21:30:50 INFO Utils: Fetching spark://localhost:49382/jars/sparklyr-2.4-2.11.jar to /private/var/folders/x3/lznkklbd17n2ljvwmyg8g1vc0000gn/T/spark-35ef4ad9-7d45-45fd-991e-2ff23426b5dd/userFiles-afa857fe-fd71-4843-883d-8a30e82fb5eb/fetchFileTemp1032874535631325098.tmp
21/03/16 21:30:50 INFO Executor: Adding file:/private/var/folders/x3/lznkklbd17n2ljvwmyg8g1vc0000gn/T/spark-35ef4ad9-7d45-45fd-991e-2ff23426b5dd/userFiles-afa857fe-fd71-4843-883d-8a30e82fb5eb/sparklyr-2.4-2.11.jar to class loader
21/03/16 21:30:51 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 1542 bytes result sent to driver
21/03/16 21:30:51 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 293 ms on localhost (executor driver) (1/1)
21/03/16 21:30:51 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool 
21/03/16 21:30:51 INFO DAGScheduler: ShuffleMapStage 0 (count at utils.scala:135) finished in 0.685 s
21/03/16 21:30:51 INFO DAGScheduler: looking for newly runnable stages
21/03/16 21:30:51 INFO DAGScheduler: running: Set()
21/03/16 21:30:51 INFO DAGScheduler: waiting: Set(ResultStage 1)
21/03/16 21:30:51 INFO DAGScheduler: failed: Set()
21/03/16 21:30:51 INFO DAGScheduler: Submitting ResultStage 1 (MapPartitionsRDD[6] at count at utils.scala:135), which has no missing parents
21/03/16 21:30:51 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 7.1 KB, free 912.3 MB)
21/03/16 21:30:51 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 3.8 KB, free 912.3 MB)
21/03/16 21:30:51 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on localhost:49383 (size: 3.8 KB, free: 912.3 MB)
21/03/16 21:30:51 INFO SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:1161
21/03/16 21:30:51 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 1 (MapPartitionsRDD[6] at count at utils.scala:135) (first 15 tasks are for partitions Vector(0))
21/03/16 21:30:51 INFO TaskSchedulerImpl: Adding task set 1.0 with 1 tasks
21/03/16 21:30:51 INFO TaskSetManager: Starting task 0.0 in stage 1.0 (TID 1, localhost, executor driver, partition 0, NODE_LOCAL, 7767 bytes)
21/03/16 21:30:51 INFO Executor: Running task 0.0 in stage 1.0 (TID 1)
21/03/16 21:30:51 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks including 1 local blocks and 0 remote blocks
21/03/16 21:30:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 9 ms
21/03/16 21:30:51 INFO Executor: Finished task 0.0 in stage 1.0 (TID 1). 1818 bytes result sent to driver
21/03/16 21:30:51 INFO TaskSetManager: Finished task 0.0 in stage 1.0 (TID 1) in 55 ms on localhost (executor driver) (1/1)
21/03/16 21:30:51 INFO TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all completed, from pool 
21/03/16 21:30:51 INFO DAGScheduler: ResultStage 1 (count at utils.scala:135) finished in 0.085 s
21/03/16 21:30:51 INFO DAGScheduler: Job 0 finished: count at utils.scala:135, took 0.866750 s
21/03/16 21:30:51 INFO HiveMetaStore: 0: get_database: default
21/03/16 21:30:51 INFO audit: ugi=nathan	ip=unknown-ip-addr	cmd=get_database: default	
21/03/16 21:30:51 INFO HiveMetaStore: 0: get_table : db=default tbl=tab1
21/03/16 21:30:51 INFO audit: ugi=nathan	ip=unknown-ip-addr	cmd=get_table : db=default tbl=tab1	
21/03/16 21:30:51 INFO HiveMetaStore: 0: get_table : db=default tbl=tab1
21/03/16 21:30:51 INFO audit: ugi=nathan	ip=unknown-ip-addr	cmd=get_table : db=default tbl=tab1	
21/03/16 21:30:51 INFO AlterTableRecoverPartitionsCommand: Recover all the partitions in file:/Users/nathan/Documents/work/NE Data Ltd/projects/catalog/inst/tinytest/, /var/folders/x3/lznkklbd17n2ljvwmyg8g1vc0000gn/T/RtmpL2S7t7/file24c544fe245b
21/03/16 21:30:51 INFO AlterTableRecoverPartitionsCommand: Found 0 partitions in file:/Users/nathan/Documents/work/NE Data Ltd/projects/catalog/inst/tinytest/, /var/folders/x3/lznkklbd17n2ljvwmyg8g1vc0000gn/T/RtmpL2S7t7/file24c544fe245b
21/03/16 21:30:51 INFO AlterTableRecoverPartitionsCommand: Finished to gather the fast stats for all 0 partitions.
21/03/16 21:30:51 INFO HiveMetaStore: 0: get_database: default
21/03/16 21:30:51 INFO audit: ugi=nathan	ip=unknown-ip-addr	cmd=get_database: default	
21/03/16 21:30:51 INFO HiveMetaStore: 0: get_table : db=default tbl=tab1
21/03/16 21:30:51 INFO audit: ugi=nathan	ip=unknown-ip-addr	cmd=get_table : db=default tbl=tab1	
21/03/16 21:30:51 INFO HiveMetaStore: 0: get_table : db=default tbl=tab1
21/03/16 21:30:51 INFO audit: ugi=nathan	ip=unknown-ip-addr	cmd=get_table : db=default tbl=tab1	
21/03/16 21:30:51 INFO HiveMetaStore: 0: get_table : db=default tbl=tab1
21/03/16 21:30:51 INFO audit: ugi=nathan	ip=unknown-ip-addr	cmd=get_table : db=default tbl=tab1	
21/03/16 21:30:51 INFO HiveMetaStore: 0: alter_table: db=default tbl=tab1 newtbl=tab1
21/03/16 21:30:51 INFO audit: ugi=nathan	ip=unknown-ip-addr	cmd=alter_table: db=default tbl=tab1 newtbl=tab1	
21/03/16 21:30:51 INFO AlterTableRecoverPartitionsCommand: Recovered all partitions (0).
21/03/16 21:35:31 INFO SparkContext: Invoking stop() from shutdown hook
21/03/16 21:35:31 INFO SparkUI: Stopped Spark web UI at http://localhost:4040
21/03/16 21:35:31 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
21/03/16 21:35:31 INFO MemoryStore: MemoryStore cleared
21/03/16 21:35:31 INFO BlockManager: BlockManager stopped
21/03/16 21:35:31 INFO BlockManagerMaster: BlockManagerMaster stopped
21/03/16 21:35:31 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
21/03/16 21:35:31 INFO SparkContext: Successfully stopped SparkContext
21/03/16 21:35:31 INFO ShutdownHookManager: Shutdown hook called
21/03/16 21:35:31 INFO ShutdownHookManager: Deleting directory /private/var/folders/x3/lznkklbd17n2ljvwmyg8g1vc0000gn/T/spark-35ef4ad9-7d45-45fd-991e-2ff23426b5dd
21/03/16 21:35:31 INFO ShutdownHookManager: Deleting directory /private/var/folders/x3/lznkklbd17n2ljvwmyg8g1vc0000gn/T/spark-7cd961e9-7174-4b4c-a154-e285a3dbf7e4
