This is about the making of Nuusie's Final Fantasy XIV Chat Extractor for the ACT Plugin.. And some of the obstacles encountered along the way. Some of the obstacles were custom colours, manipulating text to look like an emote, *Luna is writing a blog post*, quotation marks.
The chat extractor can be found here: https://github.com/saaratrix/nuu-ffxiv-act-chat-extractor

Introduction

The idea came from a friend who wanted to extract their stored chat logs and the tools she found was not as trustworthy to her as something made by me. For example this tool was hosted through a web server and she didn't want to upload her chat logs there. https://github.com/ErstonGreatman/ffxiv-act-chat-extractor.

Which lead to the first obstacle that I wanted to design the tool so you can run it locally and I wanted it to be lightweight. So that if someone does look through the code they can hopefully easily see that it just does what's expected. You could add the script and css bundles inside the html page so that it's still 1 single page but the file wouldn't look as readable. Even something smaller like Svelte still has 23 kb of minified javascript.

Developing everything in 1 file meant that I started using comments to try and separate things into sections. For example like this:


// ******************
// A section!
// ******************

About the log files

The log files has a lot of text in them as the ACT Plugin is meant to store all the log data including combat data. A chat entry is stored in the log like this:
00|Timestamp|Code|Sender|Message|Hash
The Code variable says what kind of message it is. For example '000a' for Say and '000e' is Party.

Credit goes to Isalin and their work to create the regex for parsing the chat log which sped up my development. https://github.com/isalin/ACT-Log-Extractor

Manipulating the data

After parsing all the data I had to handle it since we need to present it in a readable way for the user with colours etc. The first version just listed each line with a timestamp - sender - message.

Then the second version added colours based on the Code variable and other formatting improvements.

But now came another obstacle, post processing the data to transform it based on rules. My friend wanted the text to look different based on certain rules to fit different styles of roleplaying.

Text Transformation

I had to come up with a way to transform the text based on rules and I came up with a system that works like this:

When creating the lines we get all the active transform rules.
For each line we will iterate over all active transformers.
We check if the transformer supports that line code so for example something that only works on emotes doesn't try and transform tells.
We then use regex or a custom matcher to find any text the transformer can change.
If matches are found we then transform those matches. Which often was to add a wrapping with a class to override the previous colour.

The current rules when writing this blog post are:


const transformTextRules = [
  { codes: ['001c', '001d'], name: 'Always show emote colour', regex: /.*/gm, active: true, method: transformTextToEmoteText },
  { codes: ['001c', '001d'], name: 'Quotes in emotes as say', customMatcher: getQuotationMatches, active: true, method: transformQuotesToSayOrCustomSender },
  { codes: ['*'], name: 'Asterisks as emote', regex: /\*[^\*]+\*/gm, active: true, method: transformAsterisksAsEmote },
  { codes: ['*'], name: '( Out of character )', customMatcher: getOutOfCharacterMatches, regex: /\([^\)]+\)/, active: true, method: transformOutOfCharacter },
];

What I liked about this system is that they can be run independently on the same text. Because even if you write *(( Hello ))* it will apply both transformers. First using the asterisks one changing the text into this:
*(( Hello ))*.
And then apply the out of character transformer changing the message into:
*(( Hello ))*

Quotation Transformation

But then my friend said that quoted text inside emotes should be treated as Say. FFXIV is used by many people and cultures so I wanted to right away try and write a system that supports multiple kinds of quotation marks which was quite amusing to go to wikipedia and look up the combinations. The different kinds of quotation marks can be found here: https://en.wikipedia.org/wiki/Quotation_mark#Unicode_code_point_table.

In my first approach I tried to just greedily use a regex that checks there is a quotation mark, some text then another quotation mark however that is a very naive approach that did not work at all. For example this problem came up quickly:
"That's great!".
Where it would only find the text "That'.

So after scratching my head I changed approach to mapping each different quotation mark with a potential combination of closing marks. Which gave me this list of rules:


const quotationRules = [
  { opening: '`', closing: ['`'] },
  { opening: '#039;', closing: ['#039;'] },
  { opening: '"', closing: ['"'] },
  { opening: '‚', closing: ['‘', '’'] },
  { opening: '„', closing: ['“'] },
  { opening: '‘', closing: ['’'] },
  { opening: '“', closing: ['”'] },
  { opening: '’', closing: ['’'] },
  { opening: '‹', closing: ['›'] },
  { opening: '〈', closing: ['〉'] },
  { opening: '«', closing: ['»'] },
  { opening: '《', closing: ['》'] },
  { opening: '›', closing: ['‹'] },
  { opening: '»', closing: ['«', '»'] },
  { opening: '「', closing: ['」'] },
  { opening: '『', closing: ['』'] },
];

And then use text.indexOf() to get any opening occurrences and any closing occurrences.

Changing colour in the settings

The chat extractor has settings and you can change the colour of any message type which lead me into the world of dynamically update stylesheets. However at first I thought if I should use inline style for custom colours but that wouldn't be a good approach. Biggest reason was that it would mean that the output would have to be regenerated when a colour is updated. Which would use a lot of extra CPU and lag. So I decided to try and dynamically update the stylesheets.

I had never dynamically updated stylesheets before and I had to learn how that works but it was quite straightforward after looking at the CSSStyleSheet MDN Web Docs. To dynamically change the stylesheets I used this code and call it whenever a colour is updated.


// get the stylesheet, styleSheets[0] is style.css
// styleSheets[1] is the second style tag.
const style = document.styleSheets[1];

// Clear all the old rules, before doing this I ended up with hundreds of the same rules with slightly different colours :)
while (style.cssRules.length) {
  style.removeRule(0);
}
// Use an incrementing index because the rules are bound by indices.
let index = 0;

// Then just first add the message types (say, emote, party etc)
for (const messageType of messageTypesAsArray) {
  const css = `.${messageType.class} { color: ${messageType.color}; }`;
  style.insertRule(css, index++);
}

// Then add custom senders if the user want's a custom sender colour instead of message type colour.
for (const sender of customSenderColours) {
  const css = `.${sender.class} { color: ${sender.customColor} }`;
  style.insertRule(css, index++);
}

Deleting lines

There was another feature request to delete whole lines. Which turned out to also be simpler than I thought. A bit simplified but if we use the result from document.getSelection() and the focusNode and anchorNode. We can then get the lines and then just iterate from start to end line and delete those lines. Here's the code to remove selected lines.


/**
 *
 * @param {KeyboardEvent} event
 */
function tryRemoveSelectedText(event) {
  if (event.key !== 'Delete' && event.key !== 'Backspace') {
    return;
  }

  const selection = document.getSelection();
  if (!selection.rangeCount) {
    return;
  }

  const range = selection.getRangeAt(0);
  const outputElement = document.getElementById('output');
  const errorElement = document.getElementById('error');
  if (range.commonAncestorContainer !== outputElement && !outputElement.contains(range.commonAncestorContainer)) {
    errorElement.innerText = `Can't remove selected text because it's not only the output.`;
    return;
  }

  errorElement.innerText = '';
  const focusLine = getLineElementFromSelectionNode(selection.focusNode);
  const anchorLine = getLineElementFromSelectionNode(selection.anchorNode);

  if (!focusLine || !anchorLine) {
    return;
  }

  let focusLineNumber = parseInt(focusLine.dataset['id'], 10);
  let anchorLineNumber = parseInt(anchorLine.dataset['id'], 10);

  if (isNaN(focusLineNumber) || isNaN(anchorLineNumber)) {
    return;
  }

  // focus or anchor node can be from either direction depending if you drag left or right with the selection.
  const from = focusLineNumber <= anchorLineNumber ? focusLine : anchorLine;
  const to = focusLineNumber >= anchorLineNumber ? focusLine : anchorLine;
  let removeLines = false;

  // There can be hidden lines so we need to get them index by index.
  for (let i = outputElement.children.length - 1; i >= 0; i--) {
    const child = outputElement.children[i];
    if (!child.classList.contains('line')) {
      continue;
    }

    if (child === to) {
      removeLines = true;
    }

    if (removeLines) {
      const id = child.dataset['id'];
      const index = currentParsedLines.findIndex(l => l.id === id);
      currentParsedLines.splice(index, 1);
      outputElement.removeChild(child);
    }

    if (child === from) {
      break;
    }
  }

  generateOutput(currentParsedLines);
}

function getLineElementFromSelectionNode(node) {
  while (node.parentNode) {
    if (node.classList?.contains('line')) {
      return node;
    }

    node = node.parentNode;
  }

  return undefined;
}

Saving the output

The last part was to allow users to save the output which thanks just copying the CSS & HTML the output became WYSIWYG. To save an HTML file I added a blob download URL to an anchor tag.


function generateSaveAsHTML() {
  const outputElement = document.getElementById('output');
  const saveAsHTMLAnchor = document.querySelector('.save-as-html');
  const output = getHTMLOutput(logFilename, outputElement.innerHTML);
  const blob = new Blob([output]);
  const url = window.URL.createObjectURL(blob);
  saveAsHTMLAnchor.href = url;
  saveAsHTMLAnchor.download = logFilename + '.html';
  // For cleaning up blobs to not leak memory
  existingObjectURLs.push(url);
}

However originally I was doing it every time the output was generated which was not good because it slowed down the output generation. So I changed it to only generate the blob when the anchor tag is hovered or clicked.

Afterwords

My friend has been happy with it so far so that's nice to hear. The conversation went something like this:

"Hey I want a chat extractor!"
- "I can look into it, ooh it looks quite simple in the end"
"Then add these features!"

Saaratrix's blog

Making a FFXIV ACT Chat extractor