JsPath.json.put writing to an object in a nested array exception: expected KeyPathNode

[noisypants]

Play JSON Version (2.5.x / etc)

2.5.15

API (Scala / Java / Neither / Both)

Scala (didn't test Java)

Operating System

Window 10

JDK

java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

Library Dependencies

Expected Behavior

I would expect to use a JsPath.json.update and JsPath.json.put to update a single field within an object within a nested array. (See code below)

Actual Behavior

When attempting to put a new field in a JsObject that's inside an array, a RuntimeException is generated.
The following code throws an exception:

expected KeyPathNode
java.lang.RuntimeException: expected KeyPathNode
	at play.api.libs.json.JsPath$.step$1(JsPath.scala:142)
	at play.api.libs.json.JsPath$.step$1(JsPath.scala:141)
	at play.api.libs.json.JsPath$.play$api$libs$json$JsPath$$buildSubPath$1(JsPath.scala:147)
	at play.api.libs.json.JsPath$$anonfun$createObj$1.apply(JsPath.scala:152)
	at play.api.libs.json.JsPath$$anonfun$createObj$1.apply(JsPath.scala:150)
	at scala.collection.IndexedSeqOptimized$class.foldl(IndexedSeqOptimized.scala:57)
	at scala.collection.IndexedSeqOptimized$class.foldLeft(IndexedSeqOptimized.scala:66)
	at scala.collection.mutable.WrappedArray.foldLeft(WrappedArray.scala:35)
	at play.api.libs.json.JsPath$.createObj(JsPath.scala:150)
	at play.api.libs.json.PathReads$$anonfun$jsPut$1.apply(JsConstraints.scala:63)
	at play.api.libs.json.PathReads$$anonfun$jsPut$1.apply(JsConstraints.scala:63)
	at play.api.libs.json.Reads$$anon$8.reads(Reads.scala:126)
	at play.api.libs.json.PathReads$$anonfun$jsUpdate$1$$anonfun$apply$11.apply(JsConstraints.scala:72)
	at play.api.libs.json.PathReads$$anonfun$jsUpdate$1$$anonfun$apply$11.apply(JsConstraints.scala:72)
	at play.api.libs.json.JsResult$class.flatMap(JsResult.scala:99)
	at play.api.libs.json.JsSuccess.flatMap(JsResult.scala:9)
	at play.api.libs.json.PathReads$$anonfun$jsUpdate$1.apply(JsConstraints.scala:72)
	at play.api.libs.json.PathReads$$anonfun$jsUpdate$1.apply(JsConstraints.scala:69)
	at play.api.libs.json.Reads$$anon$8.reads(Reads.scala:126)
	at play.api.libs.json.JsValue$class.validate(JsValue.scala:18)
	at play.api.libs.json.JsObject.validate(JsValue.scala:76)
	at play.api.libs.json.JsReadable$class.transform(JsReadable.scala:29)
	at play.api.libs.json.JsObject.transform(JsValue.scala:76)

  private def setValueByPath(data: JsObject, path: JsPath, value: JsValue) : JsObject = {
     data.transform(__.json.update(path.json.put({ value }))) match {
      case s: JsSuccess[JsObject] => s.get
      case e: JsError => throw new IllegalArgumentException("Failed")
    }
  }
  val data = Json.obj(
    "array" -> Json.arr(Json.obj())
  )
  val newData = setValueByPath(data, __ \ "array" \ 0 \ "field", Json.toJson("arrayField"))

Reproducible Test Case

package jspathtest
import org.scalatest._
import play.api.libs.json._

class TestJsPath extends FlatSpec with Matchers {

  "A JsPath" should "allow setting a value in a nested branch in array" in {

    val data = Json.obj(
      "array" -> Json.arr(Json.obj())
    )
    val newData = setValueByPath(data, __ \ "array" \ 0 \ "field", Json.toJson("arrayField"))
    newData should be (Json.obj("array" -> Json.arr(Json.obj("field" -> "arrayField"))))
  }

  it should "allow setting a value in a root field" in {

    val data = Json.obj(
      "array" -> Json.arr(Json.obj())
    )
    val newData = setValueByPath(data, __ \ "field", Json.toJson("field"))

    newData should be (Json.obj("field" -> "field", "array" -> Json.arr(Json.obj())))
  }


  it should "allow setting a value in a nested obj field" in {

    val data = Json.obj(
      "other" -> Json.obj()
    )
    val newData = setValueByPath(data, __ \ "other" \ "field", Json.toJson("otherField"))
    newData should be (Json.obj("other" -> Json.obj("field" -> "otherField")))
  }

  private def setValueByPath(data: JsObject, path: JsPath, value: JsValue) : JsObject = {
    data.transform(__.json.update(path.json.put({ value }))) match {
      case s: JsSuccess[JsObject] => s.get
      case e: JsError => throw new IllegalArgumentException("Failed")
    }
  }
}

[noisypants]

After further looking into this, it seems that the JsPath.createObj method does not support IdxPathNode elements in the path. Can someone who knows this code please confirm, and maybe offer a workaround?
Thanks

[gmethvin]

@noisypants You're correct, this is a bug in JsPath.createObj. I believe it needs to be rewritten in order to deal with types other than objects. Currently we expect to be able to use JsObject.deepMerge to merge all the sub-trees from the paths at once. This doesn't work for arrays since we don't merge arrays in deepMerge; for array keys we simply replace the array.

I think the only way to fix this is to update the implementation in createObj to actually do the merging itself rather than attempting to delegate to deepMerge. @cchantep what do you think?

[rators]

any progress on this issue?

[agolovenko]

any update?

[agolovenko]

it is still a thing after 3 years

[rcongiu]

I am hitting the same issue

[sachag678]

Any thoughts on how to work around this issue? Even the suggested other libs such as play-json-zipper are no longer being maintained.