language: verbose mode

[callmehiphop]

We've received some feedback for our Natural Language implementation from the API team. They have some concerns about our verbose option.

If you're unfamiliar with said option, it essentially collapses data that was deemed non-crucial. Using the following text, please refer to the examples below - Google is an American multinational technology company specializing in Internet-related services and products.

Non-verbose Entities

entities = {
  organizations: [
    'Google'
  ],
  places: [
    'American'
  ]
}

Verbose Entities

entities = {
  organizations: [
    {
      name: 'Google',
      type: 'ORGANIZATION',
      metadata: {
        wikipedia_url: 'http: *   //en.wikipedia.org/wiki/Google'
      },
      salience: 65.137446,
      mentions: [
        {
          text: {
            content: 'Google',
            beginOffset: -1
          }
        }
      ]
    }
  ],
  places: [
    {
      name: 'American',
      type: 'LOCATION',
      metadata: {
        wikipedia_url: 'http: *   //en.wikipedia.org/wiki/United_States'
      },
      salience: 13.947371,
      mentions: [
        {
          text: {
            content: 'American',
            beginOffset: -1
          }
        }
      ]
    }
  ]
}

I believe the intention for verbose was to supply the user with the information returned from the API that covered the most common use cases, however we may actually be supplying the user with too little information making the non-verbose mode impractical. I'd like to use this issue to open a line of dialogue to see if we can improve our implementation in this area and to address any concerns from the API team.

[callmehiphop]

/cc @jgeewax @omaray @bjwatson @jmdobry @stephenplusplus @monattar

[jmdobry]

Wait, slow down. How did you do that accordion thing?

[bjwatson]

@callmehiphop Do you have a PR for the NL work so far?

[callmehiphop]

@jmdobry details! (@stephenplusplus showed me that one)

@bjwatson I still need to send it through! However this issue shouldn't affect the outcome of that PR, we agreed that this was sorta outside the scope of what we should try and accomplish.

[bjwatson]

@callmehiphop Agreed.

[stephenplusplus]

My stance is to keep the verbose option. Our job is to simplify the API response, which has to cater to every possible user and use case and make it apply for the most common user and use case. The verbose option allows the user to get more, should they need its. And in all cases, they can see the raw API response which doesn't have any of our decorations or simplifications.

We don't want to misrepresent the API from the client library, but we do want to make it as approachable as possible. As the user's needs change, they can choose the response that is the most useful to them.

[monattar]

We understand that the library team's goal is to hide some complexities from the user, however, I argue that in this particular case, you're hiding the wrong complexity. We, in the API team, along with a large team of NL researchers, PMs, and API reviewers have distilled a broad set of possible information that we can return to the user to a small and crisp set. The final API that we've released is what we believe our users want to see. The additional simplification of that, without us reviewing, is completely at odds with all we've done.
The purpose of manual client libraries is to hide the complexities that get in the way of the user getting the response that they need easily. A well-designed API can and must already produce as crisp a response as possible, which we argue we already do. There are places, however, where a client library can help and the API can't do well. A good example in NL is hiding the complexity of encoding_type, which is the exact right complexity to hide and completely within the scope and power of client libraries (and unfortunately it's not done). Other good examples are hiding retry complexities, or authentication complexities (which the client libraries already do).

[jmdobry]

FWIW, in my personal use of the Translate, NL, Speech, and Vision APIs via google-cloud-node, I find myself always passing verbose: true.

[stephenplusplus]

For the rest of our APIs, we are much more than a transport layer that makes authentication easy and retries unnoticeable. Those are "musts", for sure, but they aren't our only purpose of being. If we were to just wrap the raw API, this handwritten client layer wouldn't be necessary at all. The user could use one of the auto-generated alternatives.

This discussion about NL is no different than any other discussion with another API. It's clear that you've had long discussions about what makes a proper API request and response. What I look for when wrapping an API is what value we can additionally provide to the user. And, even though it might be less, sometimes, it's exactly what they want.

The example from the first post:

Before

entities = {
  organizations: [
    {
      name: 'Google',
      type: 'ORGANIZATION',
      metadata: {
        wikipedia_url: 'http: *   //en.wikipedia.org/wiki/Google'
      },
      salience: 65.137446,
      mentions: [
        {
          text: {
            content: 'Google',
            beginOffset: -1
          }
        }
      ]
    }
  ],
  places: [
    {
      name: 'American',
      type: 'LOCATION',
      metadata: {
        wikipedia_url: 'http: *   //en.wikipedia.org/wiki/United_States'
      },
      salience: 13.947371,
      mentions: [
        {
          text: {
            content: 'American',
            beginOffset: -1
          }
        }
      ]
    }
  ]
}

After

entities = {
  organizations: [
    'Google'
  ],
  places: [
    'American'
  ]
}

Here's what I know about users:

• We don't know exactly what they're doing
• They like things as simple as possible
• They want all available information, when they want it

That's why I don't make the assumption that every use case needs to know the Wikipedia URL and text content offset. I think what they want to know, when they send in a block of text to this method called detectEntities(), is a grouped set of "entities". And, because I'm not absolutely sure that's what they want, they can tell us they need more information by setting verbose: true. And in the case that's still not what they want, they can access the apiResponse property that their callback receives.

[monattar]

My argument is not questioning the value of client libraries. I'm 100% certain that some APIs will be very hard to use without manual client libraries. My argument is that ML APIs and in particular NL API here, at its current stage and form, is too simple to need additional response manipulation. The form of our output is the result of customer feedbacks, EAPs, and PM/eng experience in the NLU area (good example is wiki URL that our customers asked many times and you're hiding as unnecessary to show). So, the effort of the library team is not well-spent to further wrap this library. It's already simple enough for the purposes that our users have communicated to us. The only place where I see complexity could be hidden is around encoding type.

This may no longer be true in the near future, in which case, we'll be more than happy to get manual libraries. At this current point though, we should easily be able to survive with auto-generated.

[stephenplusplus]

I didn't choose to hide any response item because it isn't useful, it's simply because a user can still have a functional application using the NL API without it. It's a matter of having simple defaults that represent the most likely thing the user wanted to know. That assessment varies from user to user, but it's a certainty without the items in the default response, our response wouldn't be very useful at all. That's the only distinction, and it's only so that our users have more readable code. Returning a large object wouldn't be user-friendly in comparison to how the rest of our library operates.

[jmdobry]

the most likely thing the user wanted to know

I think what @monattar is saying is that the most likely thing the user wants to know was already determined, as she said, by "a large team of NL researchers, PMs, and API reviewers" to be "a small and crisp set" distilled from a broad set of possible information, leading to the current NL API.

Was there any study or analysis done to determine that Node.js users in particular want an even smaller set of results than what the NL API returns?

I personally am not super familiar with how people tackle Machine Learning tasks in Node.js, but if it is primarily to consume models in web apps than perhaps the ultra lean response returned by google-cloud-node is more appropriate? On the other hand it seems that at minimum you would need the confidence values to use the results in your app's own heuristics.