Control via sirs / Siri shortcuts (Pengsupport for Siri shortcuts?)

[Rainer M Krug]

Ok, so I had some fun with Siri. Mostly because of her poor accent when
reading the English responses in German...

One major flaw I see in your code: you expect players to be in sync
gropus. Without a sync group the script would fail badly. Line 597 would
crash the script.

After I added a condition in front of it (if (result.syncgroups_loop))
it did work. Though I'd often get errors due to unknown sync groups. Eg.
"undefined now paused".

Overall I'd need to spend a lot of effort to localize this. Which brings
me to the next question: what's a usable workflow? I didn't find any
search feature in scriptable's editor, nor line numbers. Which makes
navigating difficult. How did you write all that code?
--

Michael

[rkrug]

Control via sirs / Siri shortcuts (Pengsupport for Siri shortcuts?)

Hi

Now that we have Siri shortcuts in iOS, is there a chance that I could control the players via Siri? A start would be play/pause/stop, volume up and down, switch off.

Anybody done this already?

Along these lines, would it be possible to include these shortcuts in Peng?

Cheers,

Rainer

[pippin]

Yes, working on it

[rkrug]

Yes, working on it

Any news on this front?

[pippin]

Any news on this front?

No, not really. For now I moved all new functionality to the iPeng redesign. Doesn’t make sense to have to write stuff twice...

[thouky]

Siri Squeezebox Control

Now that we have Siri shortcuts in iOS, is there a chance that I could control the players via Siri? A start would be play/pause/stop, volume up and down, switch off.

Anybody done this already?

Now that the iOS 13 version of Shortcuts allows interaction within Siri, including on HomePods, it seemed worth giving this a go. I decided to implement most of it as a JavaScript script running in Simon Støvring's Scriptable app (https://scriptable.app/), with just the initial Siri interaction in Shortcuts.

In addition to play/pause/stop it also supports turning players on/off, changing volume, selecting playlists/genres/artists/albums/tracks, and asking what’s playing.

The script and installation/usage instructions can be found at:

https://github.com/thoukydides/siri-squeezebox-control

[chill]

Excellent - thanks for doing this. I jumped on it straight away, but I must have missed a step:

"OK. Something unexpected happened. (Dictation is not supported in Siri.)"

[thouky]

Excellent - thanks for doing this. I jumped on it straight away, but I must have missed a step:

"OK. Something unexpected happened. (Dictation is not supported in Siri.)"

It sounds like you are attempting to run the Scriptable script directly from Siri, instead of using the Shortcuts app to request input. Unfortunately that does not work (unless you also set Always Run in App, but that breaks the Siri experience); see https://talk.automators.fm/t/dictation-from-siri/3617.

Instead of using the Add to Siri within Scriptable you need to install or create a shortcut within the Shortcuts app that acts as an intermediary to act for the dictation. Step 2 of the installation instructions on https://github.com/thoukydides/siri-squeezebox-control is to install:

https://www.icloud.com/shortcuts/4b66dc7c872747d48228d204e58c8021

If you used the Add to Siri option within Scriptable before installing the Shortcut then that will have created a shortcut that just runs the Scriptable script. If you then installed the proper Shortcut then the name would have clashed, so the app would have renamed it automatically to be unique. In this case you will need to delete the wrong shortcut and rename (or reinstall) the correct one.

If you still can’t get it to work then try running the Scriptable script directly within its app, and the shortcut directly within the Shortcuts app. Both of these should work, albeit without the benefits of Siri.

[mherger]

Control via sirs / Siri shortcuts(Pengsupport for Siri shortcuts?)

This sounds very interesting! I hadn't seen scriptable before...

Alas, at this point it looks as if I had to translate most of your code
to German, otherwise it wouldn't work, correct? I'm even having problems
making Siri understand "Squeezebox". She's always making "Squeeze Box"
out of it. Maybe I'd have to register the activity under that term
instead. I'll play with it.

--

Michael

[thouky]

This sounds very interesting! I hadn't seen scriptable before...

Scriptable is quite a powerful tool. It is a lot easier doing anything non-trivial in Javascript than within Shortcuts' visual programming environment. The main limitation currently is that it can only present information to Siri, not request input. Hence, any non-trivial interaction within Siri requires Shortcuts to be used to obtain the necessary input first.

Alas, at this point it looks as if I had to translate most of your code
to German, otherwise it wouldn't work, correct?

Yes, I'm afraid that all of the language processing in the script is currently just for English. My foreign language skills aren't adequate for it to be worth me attempting translations...

If you don't mind the responses being in English then I think the only required changes to support input in a different language are:

• The commands and componentPatterns constants which do the main command parsing. commands provides the structure of the commands with different sentence structures, and componentPatterns provides the detailed regexps for the components of those commands and synonyms that can be used.
• The digits array which is used to convert words for single-digit numbers into their values. Siri in English tends to use words for single digit numbers, but numerals for larger values. Both need to be handled for setting volume, or for matching against playlist names like "Radio 2".
• The regexp used in doPlaylistLoad for the assignment to compoundParsed. This is used to handle search terms like "album|track by artist".

Nothing else within the script applies any meaning to the input.

Unfortunately, construction of responses is littered throughout the script. Translating all of the output would be a much bigger job.

I'm even having problems making Siri understand "Squeezebox". She's always making "Squeeze Box" out of it.

You could rename the shortcut(s) to any other trigger phrase that you like.

However, this touches on a problem with my script in its current form. Siri's speech recognition attempts to apply its own sense of context to what it hears... which the script then supplies to LMS as the search: parameter of the genres, artists, albums, and/or titles commands. Unless Siri ended up with precisely the same spelling the server obviously won't provide a match. It works most of the time for me, but there are a few artist and album names that rarely, if ever, work.

For playlists the script retrieves all of the playlist names and performs the matching on the client, which provides some scope for checking alternatives, such as allowing words to be substituted for numerals. Actually, this local matching is also applied to the other queries, mainly to restrict the results of the albums search to matches in the album title rather than those that only match a track title on the album... but it doesn't help if the required item wasn't returned by the server in the first place.

Using this approach for artists, albums, and titles would probably be too slow (and consume too much memory) for Scriptable to run in the Siri context... at least not without keeping a local cache.

[chill]

It sounds like you are attempting to run the Scriptable script directly from Siri, instead of using the Shortcuts app to request input. Unfortunately that does not work (unless you also set Always Run in App, but that breaks the Siri experience); see https://talk.automators.fm/t/dictation-from-siri/3617.

Instead of using the Add to Siri within Scriptable you need to install or create a shortcut within the Shortcuts app that acts as an intermediary to act for the dictation. Step 2 of the installation instructions on https://github.com/thoukydides/siri-squeezebox-control is to install:

https://www.icloud.com/shortcuts/4b66dc7c872747d48228d204e58c8021

If you used the Add to Siri option within Scriptable before installing the Shortcut then that will have created a shortcut that just runs the Scriptable script. If you then installed the proper Shortcut then the name would have clashed, so the app would have renamed it automatically to be unique. In this case you will need to delete the wrong shortcut and rename (or reinstall) the correct one.

If you still can’t get it to work then try running the Scriptable script directly within its app, and the shortcut directly within the Shortcuts app. Both of these should work, albeit without the benefits of Siri.

Thank you - got it now. I thought I'd followed the steps, but at one point my phone told me that it couldn't install the shortcut because it's from an untrusted source - but it seemed to skip past that when I clicked 'Add to Siri'. That was clearly the wrong thing to do.

In their infinite wisdom Apple don't let you see the option to allow untrusted shortcuts until you first install and run a shortcut from their gallery. I'm slow to criticise Apple, but that's just dumb, sorry.*

So after allowing untrusted shortcuts, I was able to complete the installation as intended. It's working well now. Siri is struggling to understand me at the moment because a cold virus this week has robbed me of my voice! But I've managed to make it do a few things, and am looking forward to doing more when my voice returns.

@Michael - a couple of times Siri has repeated the word 'Squeezebox' back to me, and she says it 'Squeezerbox'**. Maybe she'll understand you better if you say that!

EDIT:
* because the only way to sort that out is to Google the issue.
** or perhaps 'Squeezabox'

[thouky]

Thank you - got it now. I thought I'd followed the steps, but at one point my phone told me that it couldn't install the shortcut because it's from an untrusted source - but it seemed to skip past that when I clicked 'Add to Siri'. That was clearly the wrong thing to do.

In their infinite wisdom Apple don't let you see the option to allow untrusted shortcuts until you first install and run a shortcut from their gallery. I'm slow to criticise Apple, but that's just dumb, sorry.*

So after allowing untrusted shortcuts, I was able to complete the installation as intended. It's working well now.

Great to hear that you got it working. Hopefully you will find it useful once you get your voice back.

I have added a couple of notes to the installation instructions to highlight the Add to Siri and Allow Untrusted Shortcuts issues.

[mherger]

Control via sirs / Siri shortcuts(Pengsupport for Siri shortcuts?)

Thanks for the insights! I'll certainly play with it - once I find some
time to do so...

> However, this touches on a problem with my script in its current form.
> Siri's speech recognition attempts to apply its own sense of context to
> what it hears... which the script then supplies to LMS as the search:
> parameter of the genres, artists, albums, and/or titles commands. Unless
> Siri ended up with precisely the same spelling the server obviously
> won't provide a match. It works most of the time for me, but there are a
> few artist and album names that rarely, if ever, work.

It's an issue I've seen with many voice control systems: while we would
like to talk in German (or whatever other language) to those systems,
many of our artists' names are English. Even worse so album or track
titles. Those systems are still struggling dealing with mixed languages
("spiel mir Them Changes von Thundercat"). This is particularly true
when the voice recognition system doesn't know the context of the
utterance. I'll see.

--

Michael

[thouky]

One major flaw I see in your code: you expect players to be in sync
gropus. Without a sync group the script would fail badly. Line 597 would
crash the script.

Yes, all my players are always in a single sync group, so I missed that the server omits the syncgroups_loop variable in that case. It should be fixed in the latest version.

Someone else reported the same problem:

https://github.com/thoukydides/siri-squeezebox-control/issues/1

Though I'd often get errors due to unknown sync groups. Eg.
"undefined now paused".

I’ve seen that once, but haven’t been able to reproduce it.

I will investigate...

Which brings
me to the next question: what's a usable workflow? I didn't find any
search feature in scriptable's editor, nor line numbers. Which makes
navigating difficult. How did you write all that code?

I do everything on my iPad Pro with external keyboard. For debugging I make do with Scriptable’s built-in editor, but for anything non-trivial I use Textastic:

https://apps.apple.com/gb/app/textastic-code-editor-8/id1049254261

It can access the Scriptable folder on iCloud Drive directly. No need to explicitly upload/download or otherwise sync the file.

[thouky]

One major flaw I see in your code: you expect players to be in sync
gropus. Without a sync group the script would fail badly. Line 597 would
crash the script.

After I added a condition in front of it (if (result.syncgroups_loop))
it did work. Though I'd often get errors due to unknown sync groups. Eg.
"undefined now paused".

That should now be fixed by:

https://github.com/thoukydides/siri-squeezebox-control/commit/09120ea01b43449d3532e9c8f783af87ee2bb8fc

I had forgotten to set a groupname for players that weren't in a sync group.